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Geometric  Structure  of  the  Adaptive 
Controller  of  the  Human  Arm 


Reza  Sliaclmehr  and  Fordinando  A.  Mussa  Ivaldi 


Abstract 

We  investigated  how  Hie  CNS  learns  to  control  niovenienis  in  different  dynamical  conditions.  In  partic¬ 
ular.  we  considered  the  task  of  making  reaching  movements  in  the  presence  of  externally  imposed  forces 
from  a  mechanical  environment.  This  environment  was  a  force  field  produced  by  a  robot  manipulan- 
dum.  and  the  subjects  made  reaching  movements  while  holding  the  end-effector  of  this  manipulandum. 
Since  the  force  field  significantly  changed  the  dynamics  of  the  task,  subjects'  initial  movements  in  the 
force  field  were  grossly  distorted  as  compared  to  their  movements  in  free  space.  However,  with  practice, 
hand  trajectories  in  the  force  field  converged  to  a  path  very  similar  to  that  observed  in  free  space.  This 
indicated  that  for  reaching  movements,  there  was  a  kinematic  plan  independent  of  dynamical  condi¬ 
tions.  The  recovery  of  performance  within  the  changed  mechanical  environment  is  motor  adaptation. 
In  order  to  investigate  the  mechanism  underlying  this  adaptation,  we  considered  the  response  to  the 
sudden  removal  of  the  field  after  a  training  phase.  The  resulting  trajectories,  named  aff(r-(ff(rfs.  were 
approximately  mirror  images  of  those  which  were  observed  when  the  subjects  were  initially  exposed  to 
the  field.  This  suggested  that  the  motor  controller  was  gradually  composing  a  model  of  the  force  field,  a 
model  which  the  nervous  system  used  to  predict  and  compensate  the  forces  imposed  by  the  environment  . 
In  order  to  explore  the  structure  of  the  model,  we  investigated  whether  adaptation  to  a  force  field,  as 
presented  in  a  small  region,  led  to  after-effects  in  other  regions  of  the  workspace.  We  found  that  indeed 
there  were  after-effects  in  workspace  regions  where  no  exposure  to  the  field  had  taken  place,  i.e..  there 
was  transfer  beyond  the  boundary  of  the  training  data.  This  observation  rules  out  the  hypothesis  that 
the  subject's  model  of  the  force  field  was  constructed  as  a  narrow  association  between  visited  states  and 
experienced  forces,  i.e.  adaptation  was  not  via  composition  of  a  look-up  table.  In  contrast,  subjects 
modeled  the  force  field  by  a  combination  of  computational  elements  whose  output  was  broadly  tuned 
across  the  motor  state  space.  These  elements  formed  a  model  which  extrapolated  to  outside  the  training 
region  in  a  coordinate  system  similar  to  that  of  the  joints  and  muscles  rather  than  endpoint  forces.  This 
geometric  property  suggests  that  the  elements  of  the  adaptive  process  represent  dynamics  of  a  motor 
task  in  terms  of  the  intrinsic  coordinate  system  of  the  sensors  and  actuators. 

Copyright  ©  Massachusetts  Institute  of  Technology.  1993 

This  research  was  supported  in  part  by  Grants  from  the  NTH  (NSC9313  and  A R2(>7 10),  the  ON’R 
(N00014/9U/J/1946).  and  the  McDonnell  Pew  Center  for  Cognitive  Neuroscience.  This  report  describes  research 
done  at  the  Dept,  of  Brain  and  Cognitive  Sciences,  the  Center  for  Biological  and  Computational  Learning,  and 
the  Artificial  Intelligence  Laboratory  of  the  Massachusetts  Institute  of  Technology.  Support  for  CBCL  is  provided 
in  part  bv  a  grant  from  the  NSF  (ASC  9217041).  Support  for  the  laboratory's  artificial  intelligence  research  is 
provided  in  part  by  the  Advanced  Research  Projects  Agency  of  the  Dept,  of  Defense. 


1  Introduction 


Children  start  to  reach  for  objects  that  interest  them  at  about  the  age  of  3  months.  These 
goal-directed  movements  often  accompany  a  "Hailing"  action  of  the  arm.  From  a  systems  point 
of  view.  Hailing  can  be  seen  as  an  attempt  to  excite  the  dynamics  of  the  arm:  to  success¬ 
fully  make  a  reaching  movement,  the  motor  controller  needs  to  find  the  appropriate  force  so 
that  the  skeletal  system  makes  the  desired  motion.  Effectively,  this  operation  corresponds 
to  inverting  a  dynamical  transformation  that  relates  an  input  force  to  an  output  motion.  A 
controller  may  implement  this  "inverse  transformation"  via  a  combination  of  feedback  and  feed¬ 
forward  mechanisms:  usually,  the  feedforward  component  provides  some  estimate  of  the  inverse 
transformation  railed  the  "inverse  model"  or  simply  the  "internal  model"  while  the  feedback 
component  compensates  for  the  errors  of  this  estimation  and  stabilizes  the  system  about  the 
desired  behavior  (cf.  Slotine  19x5).  Therefore,  the  internal  model  refers  to  an  approximation  of 
the  inverse  dynamics  of  the  system  being  controlled.  In  the  case  of  the  infant,  action  of  flailing 
may  be  an  attempt  to  explore  this  dynamics  and  build  an  internal  model. 

During  development,  bones  grow  and  muscle  mass  increases,  changing  the  dynamics  of  the 
arm  significantly.  In  addition  to  such  gradual  variations,  the  arm  dynamics  change  in  a  shorter 
time  scale  when  we  grasp  objects  and  perform  manipulation.  The  changing  dynamics  of  the 
arm  make  it  so  that  the  same  muscle  forces  produces  a  variety  of  motor  behaviors.  It  follows 
that  to  maintain  a  desired  performance,  the  controller  needs  to  be  "robust”  to  changes  in  the 
dynamics  of  the  arm.  This  robustness  may  be  achieved  through  an  updating,  or  adaptation, 
of  the  internal  model.  Indeed,  humans  excel  in  the  ability  to  rapidly  adapt  to  the  variable 
dynamics  of  their  arm  as  the  hand  interacts  with  the  environment.  Therefore  a  task  where  the 
hand  interacts  with  a  novel  mechanical  environment  might  be  a  good  candidate  for  studying 
how  the  CNS  updates  its  internal  model  and  learns  dynamics. 

The  particular  task  which  we  have  considered  is  one  where  a  subject  makes  a  reaching  move¬ 
ment  while  the  hand  interacts  with  a  field  of  forces.  In  a  reaching  movement,  the  problem  of 
control  can  be  seen  as  one  of  transforming  information  regarding  a  target  position,  as  presented 
in  the  visual  domain,  into  a  torque  command  on  the  skeletal  system  to  move  the  hand.  This 
initially  involves  a  set  of  coordinate  transformations  (so  called  "visuo-motor  map",  cf.  Arbib 
1976):  work  of  Andersen  et  al.  (1985)  and  Soechtingand  Flanders  (1991)  suggests  that  the  tar¬ 
get  is  transformed  sequentially  from  a  ret ino- centric  vector  into  a  head-centered  and  finally  a 
shoulder-centered  coordinate  system.  According  to  Gordon  et  al.  ( 1993)  this  shoulder-centered 
vector  (representing  the  target)  undergoes  another  transformation  after  which  the  target  loca¬ 
tion  is  described  as  a  vector  with  respect  to  the  current  hand  position  (or  end  effector  position, 
for  example,  in  the  case  that  the  hand  is  holding  a  long  rod.  Lacquaniti  et  al.  1982).  At  this 
point  a  plan  is  specified,  describing  a  desired  trajectory  for  the  end-effector  to  follow:  for  un¬ 
constrained  planar  arm  movements,  there  is  strong  evidence  that  this  plan  is  a  smooth  hand 
trajectory  essentially  along  a  straight  line  to  the  target  (Morasso  1981.  Flash  and  Hogan  1985). 
The  controller,  acting  on  antagonistic  spring-like  actuators  (cf.  Bizzi  et  al.  1984.  Hogan  1985. 
Shadmehr  and  Arbib  1992).  then  attempts  to  move  the  arm  along  the  planned  trajectory. 

It  is  worth  noting  that  for  this  task,  adaptation  may  eit  her  occur  in  response  to  a  change  in 
the  visual  environment  in  which  the  target  is  presented  (von  Helmholtz  1925).  or  in  response 
to  a  change  in  the  mechanical  environment  with  which  the  hand  is  interacting  (Flash  and 
Gurevich  1992).  Therefore,  the  problem  of  adaptation  may  be  experimentally  approached  from 
two  directions: 

1.  we  may  change  the  visual  environment  so  that  subjects  have  to  modify  the  perceived 
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kint  unities  of  movement  In  changing  (lie  mapping  of  f  lie*  target  from  ego  centric  to  a  task 
based  (e.g..  hand-centered )  coordinates,  or 

2.  we  may  cliange  the  mechanical  environment  with  which  the  hand  interacts  so  that  the 
subject's  internal  model  of  the  arm  has  to  adapt  to  the  new  (lynnmi< .»  of  the  system. 

The  first  approach,  i.e..  changing  the  visually  perceived  kinematics,  has  received  much  atten¬ 
tion  because  of  the  observations  made  by  Held  and  colleagues  (Held  and  Schlank  19') 9.  Held 
1962.  Held  and  Freedman  1  )  regarding  adaptation  of  the  visuomotor  system  to  distortions 

produced  by  prism  glasses.  It  had  been  noted  that  by  wearing  prism  glasses,  the  visual  scene 
could  be  shifted,  for  example,  by  j •  degrees  laterally.  This  caused  a  cliange  in  the  kinematic 
map  relating  target  position  to  the  arm's  configuration.  With  the  glasses  on.  initially  a  subject 
would  reach  to  a  target  and  miss  it  by  .r  degrees,  but  after  some  practice,  the  subject  would 
learn  the  appropriate  kinematics  and  hit  the  target  accurately.  Predictably,  when  the  glasses 
were  removed,  the  subject  would  reach  to  a  target  and  miss  it  by  —  .r  degrees,  displaying  the 
persistence  of  the  altered  kinematic  map  (cf.  Jeannerod  19X8.  pp.  52  57).  This  behavior  has 
been  termed  an  aftt  r-f fffet  of  adaptation. 

Our  work  is  along  the  second  approach.  We  investigate  how  the  motor  control  system  re¬ 
sponds  when  the  arm's  dynamics  are  changed.  We  address  this  issue  by  developing  a  paradigm 
where  subjects  make  reaching  movements  while  interacting  with  a  virtual  mechanical  environ¬ 
ment.  From  Lackner  and  Dizio  (1992)  it  is  known  that  after-effects  exist  when  one  performs 
arm  movements  in  an  environment  where  Coriolis  forces  are  artificially  increased.  Here  we 
show  that  as  a  subject  practices  arm  movements  in  a  force  field,  the  controller  builds  an  inter¬ 
nal  model  of  that  field  and  uses  this  model  to  compensate  for  the  expected  forces  during  the 
movement.  Our  goal  is  to  understand  how  the  nervous  system  constructs  this  internal  model 
and  to  reveal  some  of  the  properties  of  the  motor  adaptive  process. 

2  Materials  and  Methods 

The  purpose  of  our  experiment  was  to  observe  how  a  subject  adapted  to  the  changed  dynamics 
of  a  reaching  task.  A  robot  manipulandum  whose  handle  was  grasped  by  the  subject  produced 
these  variable  dynamics.  A  mathematical  model  was  developed  to  provide  a  framework  for 
describing  the  process  of  adaptive  motor  control.  Both  the  experiments  and  the  modeling 
procedures  are  described  in  this  section. 

2.1  Experimental  setup 

Eight  right  handed  subjects  with  no  known  neurological  history,  ranging  in  age  from  24  to 
39.  participated  in  this  study.  A  schematic  of  the  measurement  apparatus  is  shown  in  Fig.  1: 
Subjects  were  seated  on  a  chair  that  was  bolted  onto  an  adjustable  positioning  mechanism  and 
instructed  to  grip  the  handle  of  a  robot  manipulandum  with  their  right  hand.  Their  shoulder 
was  restrained  by  a  harness  belt,  their  right  upper-arm  was  supported  in  the  horizontal  plane 
by  a  rope  attached  to  the  ceiling. 

The  manipulandum  is  a  two  degree  of  freedom,  lightweight,  low  friction  robot  (Faye  1986) 
with  a  six-axis  force-torque  transducer  (Lord  F/T  sensor)  mounted  on  its  end-effector  (the 
handle).  Two  torque  motors  (PMI  Corp.).  mounted  on  the  base  of  the  robot,  are  connected 
independently  to  each  joint  via  a  parallelogram  configuration.  Position  and  velocity  measure¬ 
ments  are  made  using  two  optical  encoders  and  tachometers,  respectively,  mounted  on  the  axes 
of  the  mechanical  joints.  The  apparatus  includes  a  video  display  monitor  mounted  directly 
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Figure  1:  Sketch  of  the  manipulanduin  and  the  experimental  setup.  Planar  arm  movements  were  made 
by  the  subject  while  grasping  the  handle  of  the  manipulanduin.  A  monitor,  placed  directly  in  front  of 
the  subject  and  above  the  manipulanduin  (not  shown),  displayed  the  location  of  the  handle  as  well  as 
targets  of  reaching  movements.  The  manipulandurn  had  two  torque  motors  at  its  base  which  allowed  for 
production  of  a  desired  force  field. 


above  the  base  of  the  robot  (approximately  at  eye  level  with  the  subject).  This  was  used  to 
display  the  position  of  the  robot's  handle  and  give  targets  for  reaching  movements. 

2.2  Experimental  procedures 

Each  subject  participated  in  a  preliminary  training  phase  where  the  task  was  to  move  a  cursor 
to  a  target.  The  cursor  was  a  square  of  size  2x2  mm2  on  a  computer  monitor  and  indicated 
the  position  of  the  handle  of  the  manipulandurn.  Targets  were  specified  by  a  square  of  size  XxX 
mm2.  The  task  was  to  move  the  manipulandurn  so  as  to  bring  the  cursor  within  the  target 
square. 

Movements  took  place  in  two  regions,  each  of  the  size  15x15  cm2.  The  position  of  these 
regions  is  shown  in  Fig.  2.  where  they  are  labeled  as  the  "left"  and  “right"  workspaces.  In 
order  to  avoid  inertial  artifacts  associated  with  changing  the  operating  configuration  of  the 
robot,  workspaces  were  selected  by  moving  the  subject  with  respect  to  the  robot. 

Starting  from  the  center  of  a  workspace,  a  target  at  a  direction  randomly  chosen  from  the 

set  {0°.45° . 325°}  and  at  a  distance  of  10  cm  was  presented.  After  the  subject  had  moved 

to  the  target,  the  next  target,  again  chosen  at  a  random  direction  and  at  10  cm  was  presented. 
A  target  set  consisted  of  250  such  sequential  reaching  movements.  All  targets  were  kept  with 
in  the  confines  of  the  15x15  cm  workspace.  The  targets  represented  a  pseudo-random  walk. 

In  some  cases,  the  manipulandurn  was  programmed  to  produce  forces  on  the  hand  of  the 
subject  as  the  subject  performed  reaching  movements.  These  forces,  indicated  by  the  vector  /. 
were  computed  as  a  function  of  the  velocity  of  the  hand: 

f=Bx  (1) 

where  i  was  the  hand  velocity  vectoi.  and  B  was  a  constant  matrix  representing  visc^.-uty  of 
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Figure  2:  Configurations  of  a  model  two  joint  arm.  representing  typical  kinematics  of  the  human  arm. 
at  two  workspace  locations  where  reaching  movements  were  performed.  Typical  shoulder  and  elhow 
angles  at  these  two  workspaces  were  15  and  1U0  degrees  at  right  and  (>0  and  1-1*)  degrees  at  left. 


the  imposed  environment  in  endpoint  coordinates.  In  particular,  we  chose  H  to  be: 


B  = 


-10.1  -11.2 

-11.2  11.1 


iN’.sec/m 


Using  this  matrix,  the  forces  defined  by  Eq.  (1)  may  be  shown  as  a  field  over  the  space  of 
hand  velocities  (Fig.  3A).  For  example,  as  a  subject  made  reaching  movements  in  this  field,  the 
manipulandum  produced  forces  shown  in  Fig.  313  (here  we  have  assumed  that  the  movements 
are  minimum  jerk,  as  specified  by  Flash  and  Hogan  (1985).  with  a  period  of  0.5  seconds). 

Note  that  in  the  field  defined  by  Eq.  (1).  forces  which  act  on  the  hand  are  invariant  to 
the  location  of  the  workspace  in  which  a  movement  is  done.  i.e..  the  forces  are  identical  in  the 
left  and  right  workspaces  of  Fig.  2.  Therefore,  we  say  that  the  force  field  defined  in  Eq.  (1)  is 
translation  invariant  in  endpoint  coordinates. 

In  some  cases,  a  different  kind  of  a  force  field  was  produced  by  the  manipulandum.  one 
which  was  not  translation  invariant  in  endpoint  coordinates.  This  field  was  represented  as  a 
function  of  the  angular  velocity  of  the  subject's  shoulder  and  elbow  joints  during  the  reaching 
movements: 

r=Wq  (2) 

where  r  was  the  torque  vector  acting  on  the  subject’s  shoulder  and  elbow  joints,  q  was  the 
subject's  joint  angular  velocity,  and  W  was  a  constant  matrix  representing  viscosity  of  the 
imposed  environment  in  joint-coordinates  of  the  subject.  We  say  that  the  field  described  by 
Eq.  (2)  is  translation  invariant  in  joint-coordinates.  Indeed,  note  that  the  torque  field  in  Eq. 
(2)  is  equivalent  to  the  following  force  field  (i.e..  forces  acting  on  the  hand): 

f=(j{q)TY'wq  (3) 

where  ./(</)  =  0x/0q.  is  the  configuration-dependent  Jacobian  of  the  configuration  mapping 
from  q  to  x.  and  the  superscript  T  indicates  the  transpose  operation.  Because  the  Jacobian 
changes  as  a  function  of  the  angular  position  of  the  limb.  /  varies  depending  on  the  workspace 
where  a  reaching  movement  is  performed.  In  particular,  we  chose  IF  so  that  the  force  field 
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Figure  3:  An  environment  as  described  by  the  force  field  in  Eq.  (1).  A:  The  force  field.  B:  Forces 
acting  on  the  hand  during  simulated  center  out  reaching  movements.  Movements  are  simulated  as  being 
minimum- jerk  with  a  period  of  0.‘>  sec  and  amplitude  of  10  cm. 


which  resulted  from  Eq.  (3)  at  the  right  workspace  was  almost  identical  to  the  field  produced 
by  Eq.  ( 1 ).  To  accomplish  this,  the  matrix  IF  was  calculated  for  each  subject  as: 

H  =  •/,[  B  J0 


where  ./0  is  the  Jacobian  evaluated  at  the  center  of  the  right  workspace.  For  a  typical  subject, 
we  derived  the  following  IT  matrix: 


n  = 


0.0  -1.7 

-1.7  -1.7 


N.  m. sec/ rad 


When  the  above  joint-viscosity  matrix  was  used  to  define  an  environment,  the  resulting  force 
field  depended  upon  the  position  of  the  workspace  where  movements  were  being  made.  At  the 
right  workspace,  this  field  ( Eq.  3)  was  almost  identical  to  that  produced  by  Eq.  ( 1 )  (a  correlation 
coefficient  of  0.99.  see  Appendix  I).  However,  at  the  left  workspace,  the  forces  produced  by  Eq. 
(3)  were  substantially  uncorrelated  (nearly  orthogonal)  to  that  of  Eq.  (1).  The  force  field 
produced  by  Eq.  (3)  is  plotted  for  movements  in  the  left  workspace  in  Fig.  4A.  Fig.  4B  shows 
the  forces  acting  on  the  hand  for  typical  reaching  movements. 

We  trained  subjects  with  either  the  end-point  or  the  joint  translation  invariant  fields  at  the 
right  workspace.  Subsequently,  we  tested  them  in  the  field  they  had  not  been  trained  on  at 
the  left  workspace.  Hence,  we  defined  two  distinct  groups  of  subjects.  Those  in  Group  1  were 
exposed  to  a  force  field  which  was  translation  invariant  with  respect  to  the  position  of  the  hand 
(Eq.  1).  Subjects  of  Group  2  were  exposed  to  a  force  field  which  was  translation  invariant 
with  respect  to  the  angular  position  of  the  subject's  joints  ( Eq.  3). 

Our  first  objective  was  to  compare  movements  during  conditions  of  no-visual  feedback  before 
and  during  the  initial  exposure  to  a  field.  For  IS  randomly  chosen  members  of  the  target  set. 
heretofore  referred  to  as  the  no-vision  target  set.  the  cursor  position  during  the  movement  was 
blanked,  removing  visual  feedback  during  the  reaching  period.  For  the  remaining  members  of 


■) 


1 


Hand  x-velocity  (m/s)  Displacement  (mm) 


Figure  4:  An  environment  described  by  t he  field  in  Kq.  (:{).  plotted  as  it  would  appear  in  the  left 
workspace  of  Fig.  2.  A:  The  force  field  B:  Forces  acting  on  the  hand  while  making  reaching  movements 
in  the  left  workspace  of  Fig.  2  from  the  center  to  targets  about  tin*  circumference  of  a  circle.  Movements 
are  simulated  as  being  minimum-jerk  with  a  period  of  0.5  sec  and  amplitude  of  10  cm. 


the  target  set.  hand  position  was  shown  continuously  lo  the  subject.  Initially,  we  quantified 
the  performance  in  a  null  field,  i.e..  with  the  torque  motors  turned  off.  by  presenting  a  target 
set  in  the  right  workspace.  Upon  completion,  the  hand  was  moved  to  the  left  workspace 
and  another  target  set  presented.  These  hand  trajectories  during  the  no-vision  target  sets 
represented  baseline  performance  of  the  subjects  in  the  null  field. 

Following  this,  the  hand  was  returned  to  the  right  workspace  and  the  target  set  was  again 
presented,  except  that  for  24  randomly  chosen  members  of  the  no-vision  target  set.  the  manip- 
ulandum  produced  the  force  field  assigned  to  the  subject's  group.  For  the  remaining  targets 
of  this  set  a  null  field  was  present.  These  hand  trajectories  during  the  no- vision  target  set 
represented  baseline  performance  in  the  force  field. 

The  next  objective  was  to  observe  performance  of  the  subject  in  response  to  continuous 
exposure  to  the  force  field:  With  the  hand  at  the  right  workspace  and  with  the  manipulandum 
producing  the  force  field,  a  target  set  was  presented.  The  force  field  was  present  for  all  targets 
except  for  24  randomly  chosen  members  of  the  no- vision  target  set.  where  the  null  field  was 
present.  The  purpose  of  these  24  targets  in  the  null  field  was  to  record  any  aji(r-r jjrcts  of 
adaptation  to  the  force  field.  The  target  set  was  repeated  4  times  (total  of  1000  movements) 
while  the  manipulandum  produced  the  field.  This  provided  time  for  the  subject  to  adapt  to 
the  field. 

Having  completed  the  adaptation  phase  of  the  experiment,  the  subject's  arm  was  moved  to 
the  left  workspace  with  the  objective  of  observing  any  trails f rrifr I  after-rfffcta.  72  targets  were 
presented  sequentially  and  with  no  visual  feedback.  24  randomly  chosen  members  of  this  target 
set  were  in  a  null  field.  Another  24  randomly  chosen  members  of  this  targel  set  were  in  the 
force  field  on  which  the  subject  had  been  trained  on.  The  remaining  members  of  this  target  set 
were  in  the  force  field  which  the  subject  had  not  been  trained  on. 
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2.3  Producing  the  force  fields 

In  order  for  the  manipulandum  to  produce  a  given  force  lield.  the  microcomputer  collected 
position  and  velocity  information  from  the  manipulandum’s  joints  (represented  by  o  and  of  at 
a  rate*  of  100  Hz.  This  information  was  needed  in  order  to  convert  the  desired  midpoint  force 
field  into  the  torques  to  be  applied  by  the  motors.  To  produce  the  force  field  described  by  Eq. 
( 1 ).  we  used  the1  following  expression: 

TH  =  h<  H  hi  o 


where  r/y  is  the  torque  vector  commanded  to  the  motors,  Jn  =  Ox /Ho.  i.e..  the  Jacobian  of  the* 
robot's  kinematics,  and  o  is  the*  joint  angular  velocity  vector  of  the  manipulandum.  Note  that 
.//,*  is  a  function  of  robot  joint  angles  o.  and  from  its  definition  il  follows  that  ./•  =  JpO.  In 
order  to  prod1  tee  the  force  field  described  by  Eq.  (:f).  the  following  control  law  was  used: 

r,t  =  Jjf  l~T  M  J,{  o 

where  ./  is  the  subject's  .Jacobian  matrix  function.  Calculation  of./  required  knowledge  of 
the  subject's  arm  kinematics:  At  the  beginning  of  each  session,  we  measured  the  lengths  of 
the  subject's  upperarm  and  forearm  as  well  as  the  location  of  the  shoulder  with  respect  to  a 
fixed  point  with  respect  to  the  workspace  of  the  manipulandum.  These  data  were  sufficient  to 
provide  an  estimate  for  .1  at  each  position  of  the  hand. 

2.4  Data  analysis 

We  sampled  hand  positions  and  velocities  at  i()  msec  intervals  as  the  subject  reached  to  a 
target.  Trajectories  were  aligned  using  a  velocity  threshold  at  the  ousel  of  movement. 

In  order  to  compare  hand  trajectories,  a  technique  was  developed  which  quantified  a  mea¬ 
sure  of  correlation  between  two  sampled  vector  fields  (see  Appendix  1).  We  represented  each 
trajectory  as  a  time  series  of  velocity  vectors  (x  sampled  at  10  msec  intervals)  and  then  com¬ 
pared  the  two  resulting  vector  fields  through  a  correlation  measure.  The  same  technique  was 
also  used  to  compare  force  fields.  In  particular,  the  endpoint  viscosity  matrix  H  in  Eq.  ( 1 )  was 
chosen  such  that  when  expressed  in  terms  of  a  joint  viscosity  matrix  U’  (through  Eq.  J).  the 
two  resulting  force  fields  were  nearly  identical  at  the  right  workspace  (the  correlation  coefficient 
p  %  1 ).  while  maximally  different  at  the  left  workspace  (p  0).  Specifically,  the  two  fields  had 
a  correlation  coefficient  of  0.99  and  0.12  at  the  right  and  left  workspaces,  respectively. 

In  order  to  plot  "typical"  hand  trajectories  for  a  given  target,  we  computed  the  expected 
value  and  standard  deviation  of  the  set  of  measured  trajectories  (each  a  time  series  of  velocity 
vectors)  for  that  target.  Our  procedure  consisted  in  deriving  the  expected  value  and  standard 
deviation  of  the  set  of  measured  velocity  vectors  across  the  trajectories  at  each  instant  of  time. 
The  resulting  velocity  field  was  integrated  from  the  start  position  of  the  movement  to  produce 
the  average  ±  standard  deviation  of  the  hand  trajectories  for  a  given  target. 

2.5  Mathematical  modeling 

The  purpose  of  the  mathematical  modeling  was  to  provide  a  framework  for  describing  how  the 
subject's  motor  control  system  adapted  to  a  force  field  imposed  on  the  hand.  A  significant 
component  of  our  task  was  to  specify  what  we  meant  by  an  "internal  model"  as  it  pertained  to 
the  adaptation  process.  The  end  result  was  a  system  of  equations  which  allowed  us  to  simulate 
hand  trajectories  for  reaching  movements  before  the  subject  had  adapted  to  the  force  field,  as 


well  as  the  after-effects  when  the  subject  had  formed  an  internal  model  hut  the  external  field 
was  suddenly  removed. 

2.5.1  General  framework  for  a  model-based  adaptive  controller 

Let  us  start  by  considering  a  system’s  dynamics  in  generalized  coordinates  (for  a  review  of  the 
idea  behind  generalized  coordinates,  see  Spong  ami  Vidyasagar  I !»>!>.  p.  1  -1 1  ).  We  indicate  by 
q  a  point  in  configuration  space  (e.g..  an  array  of  joint  angles)  and  by  q  and  </  its  first  and 
second  time  derivatives.  The  dynamics  of  the  motor  control  system  coupled  (in  parallel)  with 
its  environment  can  be  described  by  the  sum  of  three  distinct  force  fields:  (  I  )  a  time  invariant 
component.  D(q.q.ij).  representing  the  forces  widt  h  depend  on  the  "passive"  or  unmodulated 
system  dynamics  (bones,  tendons,  ligaments,  connective  tissues,  etc.).1  (2)  a  time  invariant 
component .  E(q.  q.  <j).  representing  t  he  forces  which  depend  on  t  he  dynamics  of  t  he  environment . 
and  (3)  a  time  varying  component,  (’(q.q.t).  representing  the  forces  which  depend  on  the 
operation  of  the  controller.  Following  D'Alembert's  principle,  the  entire  system/en vironment 
dynamics  is  summarized  by  an  equilibrium  condition: 

D(  q.  q.ij)  +  E(  q.  q.  q)  =  (‘(q.q.t)  (  I ) 

The  force  field  represented  by  I)  is  itself  a  sum  of  inertial  and  velocity  dependent  forces.  The 
inertial  component  of  this  field  is  due  to  the  acceleration  of  the  system  and  is  defined  as  a 
vector  equal  in  magnitude  to  but  opposite  to  the  direction  of  the  rate  of  change  of  momentum 
of  the  mass  of  the  system.  If  we  restrict  our  attention  to  Newtonian  mechanical  systems,  we 
can  assume  that  inertial  forces  are  linearly  related  to  acceleration.  In  this  case,  the  field  I)  can 
be  written  as: 

D(q.q.ij)  =  I(q)  q  +  G(q.q)  (•">) 

where  /  represents  the  system's  mass  in  generalized  coordinates  (an  inertia  matrix,  which  may 
be  a  function  of  configuration),  and  G  represents  the  rest  of  the  position  and  velocity  dependent 
forces  (i.e..  Coriolis,  friction,  etc.). 

Let  us  consider  a  control  system  that  is  capable  of  guiding  a  limb  along  a  desired  trajectory 
r/*(/)  in  the  null  environment  E  =  0.  One  way  to  obtain  this  desired  tracking  behavior  is  by 
picking  the  right  hand  side  of  Eq.  (4)  to  be  an  i(l(dl  contmlUr  specified  by  l{q)qm(t)  +  G(q.q). 
This  simplifies  Eq.  (4)  to  q  =  q*(t).  from  which  it  follows  that  from  some  given  initial  position 
and  velocity,  the  system  will  follow  the  desired  trajectory.  Note  that  this  ideal  control  input 
describes  a  time  varying  force  field:  for  a  given  desired  acceleration,  a  force  vector  is  assigned 
to  each  point  in  the  state-space  of  the  system.  A’e  name  this  ideal  controller  P.  i.e.. 

P(q.q.t)  =  I(q)q~(t)  +  G(q.q).  (b) 

We  call  this  controller  "ideal'"  because  it  may  well  be  that  one  cannot  implement  its  field  using 
the  available  actuators  and  local  controllers.  However,  one  may  be  able  to  approximate  its  force 
field,  resulting  in  an  internal  model  of  the  system  dynamics.  Let  us  call  this  internal  model  P: 

V  ~  P(q.  q.  t )  (7) 

’There  is  not  a  single  component  in  the  scenario  of  biological  systems  that  may  be  properly  characterized  as 
“time-invariant"  —  bones  grow,  and  ligaments  age.  Here,  however,  we  are  primarily  concerned  with  the  •unit  of 
the  temporal  intervals  within  which  changes  occur.  In  t  his  respect  .the  neuromuscular  cont  rolling  system  operates 
on  the  scale  of  10~ ^  seconds.  In  contrast,  the  changes  in  the  unmodulated  dynamics.  D.  may  be  assumed  to 
occur  on  a  time  interval  that  is  5  or  more  orders  of  magnitude  longer. 
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where  fur  the  system  dynamics  of  Kq.  (•"»)•  with  a  null  environment.  our  internal  model  may  be 
deli ned  In  the  following  field: 

P  =  /  </'(/)  +  (• 

We  have  just  shown  that  the  internal  model  fora  second-order  time -invariant  field  D\q.  q.ij). 
is  a  f i ist  order  time-varying  field  P(r/.  </./).  In  effect,  the  internal  model  is  not  a  model  of  the 
dynamical  system,  but  a  model  of  the  ideal  controller  lor  that  dynamical  system. 

1’ nfort unately.  even  with  an  exact  model,  i.e..  if  the  similarity  in  Kq.  <  7 )  is  replaced  In 
an  equality.  the  resulting  coupled  system  will  be  unstable  about  the  desired  trajectory:  our 
controller  will  not  be  able  to  compensate  for  t  he  slightest  unexpected  change  in  initial  condi’  ions 
or  for  any  perturbation  occurring  during  the  movement.  One  way  to  overcome  this  is  to  define 
our  controller  ('  in  Kq.  (  I)  (assuming  a  null  environment  for  now)  so  that  it  combine*  the 
internal  model  of  Kq.  (7)  with  an  error  fet'd  back  system  designed  to  provide  stability  about  the 
desired  t ra jectory : 

('(</. <l.t)  =  n  -  S{q-q'(t).q-  qU))  (*> 

where  S  is  a  converging  force  field  about  the  desired  state  of  the  system  at  time  I.  i.e..  it 
has  zero  forces  only  when  both  of  its  arguments  are  zero.  It  can  be  demonstrated  that  by 
properly  selecting  behavior  of  the  field  S  about  this  null  position,  a  multi-input,  multi-output 
lion-linear  system  such  as  that  of  a  .mlti-joint  arm  will  be  stable  about  a  desired  tra  jectory 
despite  uncertainties  in  initial  position  and  errors  in  the  internal  model  (Slotine  and  I.i  1991 ). 

Now  lot  us  apply  an  environment  E  ^  0  and  consider  the  problem  of  finding  a  new  controller 
such  that  q‘(t)  is  still  the  solution  for  the  coupled  dynamics  described  by  Kq.  (  I ).  The  procedure 
is  similar  to  the  one  just  described:  ideally,  we  would  like  to  replace  the  right  hand  side  of  Kq. 
(4)  by  the  field:  Piq.q.l)  +  £(q.q.t).  where  £  is  an  ideal  control  input  chosen  such  that  the 
differential  equation  E(q.q.<j)  =  £{q.q.t)  has  a  solution  </“(/)  from  a  given  initial  position.  We 
therefore  express  the  new  controller  as: 

C(q.q.l)  =  P  +  £  -  S{q  -  q~(t).q  -  q~(/))  (ft) 

where  £{q.q.t)  is  our  model  of  the  environment,  expressed  as  a  first  order  time  varying  field: 

£  -  £(q.q.D  (10) 

Assuming  that  the  system  was  capable  of  producing  the  desired  tra  jectory  in  the  absence  of  an 
environment,  then  it  is  apparent  that  as  £  —  £.  the  coupled  dynamics  is  reduced  back  to  the 
form  of  Kq.  (4).  of  which  the  desired  trajectory  qM(l)  was  a  particular  solution. 

This  formalism  shares  a  key  concept  with  the  learning  theory  forwarder!  by  Kelso.  Saltzman 
and  coworkers  (Kelso  and  Schoner  IONS.  Saltzman  and  Kelso  19X9.  Schoner  et  al.  1092).  where 
a  motor  plan  is  achieved  through  a  change  in  the  dynamics  of  the  effector  such  that  the  new 
dynamics  have  an  "attractor"  at  the  to-be-learned  trajectory.  In  other  words,  here  we  arc 
reiterating  the  classical  paradigm  of  adaptive  control,  according  to  which  in  order  to  rope 
with  a  changing  environment  it  is  sufficient  to  add  a  component  to  the  controller  which  has 
the  specific  task  of  identifying  the  new  dynamics.  In  this  case,  the  identification  is  through 
approximation  of  a  time-varying  force  field  £.  which  represents  the  ideal  control  input  to  the 
environment.  The  goal  of  adaptation  is  to  find  £.  i.e..  the  internal  model  of  the  environment, 
such  that  the  similarity  in  Kq.  (10)  is  replaced  with  an  equality.  If  this  is  achieved,  then  all 
trajectories  that  the  controller  was  able  to  perform  in  the  null  environment  are  equally  well 
performed  in  the  non-zero  environment . 


2.5.2  A  model  of  the  controller  for  reaching  movements 

We  consider  the  hypothesis  that,  as  a  subject  interacts  with  the  environmental  lorce  field,  the 
internal  model  of  t  hat  field  ( i.e..  K<|.  10 )  becomes  more  and  more  accurate.  One  way  to  quantify 
this  gradual  development  is  through  the  notion  of  an  after-effect:  occasionally  we  removed  the 
lield  at  the  onset  of  movement  and  observed  how  the  hand  trajectories  behaved  compared  to 
wlit'ii  the  subject  had  not  been  exposed  to  the  field.  If  an  internal  model  had  formed,  then  we 
should  be  able  to  predict  a  change  in  the  hand  trajectory  bast'd  on  t  lie  simulation  ol  a  control 
system  that  attempted  to  compensate  for  t lie  expected  environmental  tones.  Note  that  this 
operation  was  made  possible  by  the  fact  that  the  force  fields  in  lap.  (  1 )  and  ( 4 )  were  zero  when 
the  hand  was  at  rest,  resulting  in  a  situation  where  the  subject  could  not  determine  before 
initiation  of  movement  it  the  held  had  been  removed. 

In  order  to  simulate  t  he  after-effect  s  of  adaptation,  we  initially  consider*  d  the  task  of  learning 
to  reach  1o  targets  in  the  field  of  lap  (  1 ).  The  skeletal  dynamics  of  lap  (■’>)  were  simulated  for 
each  subject  using  an  inertial  matrix  /(r/)  as  measured  by  Diffrieut  el  al.  (1117'').  and  given  for 
a  typical  subject  in  Table  I  (the  Coriolis  and  centripetal  forces  which  make  up  the  (i  matrix 
can  be  derived  from  the  inertia  tensor,  cf.  Slotiue  and  l.i  1991.  p.  100).  from  Kqs.  (  I)  anti  (5) 
we  had: 

l(q)  <j+  (1  ( q.  <i )  +  •/(  q )‘  B  ./( q)  <)  =  C(q.q.t)  (II) 

I’sing  the  notion  of  an  internal  model,  the  controller  ('  was  defined  for  t he  case  of  a  neuromus¬ 
cular  system  in  the  form  of  Ftp  (9): 

C(q.q.t)  =  I  ij'(l)  +  (!  +  t'  -  l\  (q  -  q"(( ))  -  l  (<)  -  q‘(l))  (  12) 

where  A  and  l  were  joint  stiffness  and  viscosity  matrices  describing  the  behavior  of  the  field 
S  in  Ftp  (9)  about  the  desired  trajectory.  For  each  subject,  we  assumed  a  joint  stiffness  matrix 
I\  as  measured  by  Mussa-Ivaldi  et  al.  (19X5).  a  joint  viscous  matrix  \  as  measured  by  Tsuji 
and  Goto  ( 1995)  (i.e..  same  orientation  and  shape  as  that  of  joint  stiffness,  scaled  by  0.15  sec), 
and  a  desired  hand  path  described  by  a  minimum  jerk  trajectory  lasting  0.05  seconds.  The 
particular  values  of  these  variables  are  summarized  in  Table  1. 

3  Results 

Our  paradigm  was  one  in  which  reaching  movements  were  made  while  the  hand  interacted  with 
a  programmable  mechanical  environment.  This  environment  was  a  programmable  force  field 
implemented  by  a  light  weight  robot  manipulandum  whose  end  effector  the  subject  grasped 
while  making  reaching  movements.  When  the  manipulandum  was  producing  a  force  field,  there 
were  forces  which  acted  on  the  hand  as  it  made  a  movement,  changing  the  dynamics  of  the 
arm.  When  the  manipulandunfs  motors  were  turned  off.  we  say  that  the  hand  was  moving  in 
a  "null  field". 

3.1  Hand  trajectories  before  adaptation 

Our  first  objective  was  *o  determine  how  an  unanticipated  velocity  dependent  field  affected 
the  execution  of  reaching  movements.  The  forces  in  t lie  field  (e.g..  K<p  I.  as  shown  in  Fig.  3A) 
vanished  when  the  hand  was  at  rest,  that  is.  at  the  beginning  and  at  the  end  of  the  movement. 
However,  as  shown  in  Figure  4B.  a  significant  force  was  exerted  midway,  when  the  hand  velocity 
was  near  maximum.  How  would  this  force  influence  the  execution  of  a  movement?  Would 
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Figure  5:  Typical  hand  trajectories  at  the  right  workspace  in  a  null  force  field  during  no  visual  feedback 
conditions.  Dots  are  10  msec  apart. 


subjects  follow  a  pre-planned  trajectory  that  was  scarcely  influenced  by  this  perturbation  or 
would  they  modify  the  movement  and  the  final  position  in  response  to  the  perturbing  force? 
To  answer  this  question,  we  compared  reaching  movements  in  the  null  field  with  those  in  a 
force  field.  Trajectories  in  the  null  field  are  shown  in  Fig.  5.  As  observed  in  previous  reports 
(Morasso  19X1.  Flash  and  Hogan.  19X5).  the  hand  path  was  essentially  along  a  straight  line  to 
the  target.  The  velocity  profile  (see  Fig.  9A)  had  one  peak,  with  approximately  equal  times 
spent  to  accelerate  and  decelerate  the  hand. 

Once  our  subjects  were  familiar  with  the  task  of  reaching  within  the  null  field,  we  begun 
to  introduce  a  force  field  in  random  trials  as  detailed  in  the  methods.  We  should  stress  that 
at  this  stage,  subjects  could  not  anticipate  the  presence  of  the  field  before  the  onset  of  the 
movement  because  the  force  field  was  not  effective  when  the  hand  was  at  rest  and  no  other 
clues  were  available.  Furthermore,  during  the  movement,  the  cursor  indicating  hand  position 
was  blanked,  eliminating  visual  feedback.  Figure  0  shows  the  hand  trajectories  (average  ± 
standard-deviation)  of  a  typical  subject  when  the  movements  were  executed  under  the  influence 
of  the  field  shown  in  Fig.  3A.  Figure  9B  shows  the  tangential  velocity  of  hand  trajectories  in  this 
field.  This  field  was  designed  to  have  opposing  effects  along  two  directions.  At  approximately 
30  and  210  degrees  the  field  produced  resisting  forces  that  opposed  movement  as  a  viscous 
fluid  would  do.  At  approximately  120  and  300  degrees  the  forces  assisted  the  movement,  thus 
producing  a  de-stabilizing  effect. 

Note  that  the  effect  of  the  field  on  the  hand  trajectory  was  quite  significant  and  may  be 
divided  into  two  parts.  In  the  first  part,  the  hand  was  driven  off-course  by  the  field  and  forced 
towards  the  unstable  direction  of  the  field.  Movements  to  target  s  at  0.  225.  270.  and  315  degrees 
are  pulled  toward  the  unstable  region  at  300  degrees,  while  movements  to  the  remaining  targets 
are  pulled  toward  the  unstable  region  at  120  degrees.  At  the  end  of  this  first  part,  the  field  had 
caused  the  hand  to  veer  off  the  direction  of  the  target  and  the  hand  decelerated  and  stopped 
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Figure  6:  Performance  during  initial  exposure  to  a  force  field.  Shown  are  hand  trajectories  to  targets  at 
the  right  workspace  while  moving  in  the  force  field  shown  in  Fig.  .1.  Movements  originate  at  the  center. 
All  trajectories  shown  are  under  no  visual  feedback  condition.  Dots  are  10  msec  apart. 

before  making  a  second  movement  to  the  target.  The  pictorial  effect  of  these  two  parts  of  the 
hand  trajectory  appeared  as  a  "hook"  that  was  oriented  either  clockwise  or  counterclockwise. 
The  orientation  and  the  overall  appearance  of  this  hook  was  found  to  depend  upon  the  position 
of  the  target  and  the  pattern  of  forces  in  the  field,  and  was  very  similar  among  the  X  subjects. 

One  may  interpret  the  hooks  shown  in  Fig.  6  as  “corrective  movements'*  that  are  generated 
to  compensate  for  the  errors  caused  by  the  unexpected  field.  In  light  of  the  fact  that  no  visual 
feedback  was  available  to  the  subjects  during  the  movements  shown  in  Fig.  6.  this  correction 
might  imply  some  explicit  reprogramming  of  the  movement  based  on  proprioceptive  information 
detecting  the  error  in  the  hand  trajectory.  Alternatively,  this  feature  of  the  trajectory  might 
be  a  byproduct  of  a  "robust"  control  system  implementing  a  single  progiri/n:  In  this  case,  the 
program  would  be  to  simply  move  the  hand  along  a  desired  trajectory  to  the  target.  The 
corrective  movements  might  result  because  of  the  natural  interaction  between  the  mechanical 
properties  of  the  arm.  as  imposed  from  the  controller,  and  the  force  field  produced  by  the 
manipulandum.  To  explore  this  scenario,  we  simulated  the  operation  of  a  controller  acting  on 
the  arm's  skeletal  system  via  antagonistic  muscles  within  the  force  field.  The  controller,  which 
is  detailed  in  the  Methods  section  (Eq.  12).  was  designed  based  on  the  assumption  that  the  goal 
was  to  move  the  limb  along  a  smooth,  straight  line  trajectory  to  the  target.  We  further  assumed 
that  the  controller  had.  through  years  of  practice,  composed  an  accurate  internal  model  of  the 
skeletal  dynamics.  This  internal  model  was  used  to  activate  the  muscles  and  produce  torques 
such  that  the  system  defined  by  Eq.  (4)  (and  assuming  a  null  field)  had.  in  ideal  conditions 
(i.c..  with  a  perfect  internal  model  of  the  skeletal  dynamics),  a  solution  identical  to  the  desired 
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Table  1:  Mechanical  parameters  of  the  simulated  human  arm 


Cpperarm 


Forearm 


Stiffness 

Viscosity 


mass 

center  of  mass 

inertia 

length 

mass 

center  of  mass 


inertia 

iengt  li 

’  -15 

-G 

-6 

-16 

-2.3 

-0.9 

-0.9 

-2.4 

1 .93  kg 
0.105  m 
0.0141  kg. m2 
0.33  m 
1.52  kg 
0.19  m 
.0188  kg. m 2 
0.44  m 

N. m/rad 
N.m.sec/rad 


trajectory.  However,  recognizing  that  there  might  he  errors  in  this  internal  model,  the  controller 
used  the  viscoelastic  properties  of  the  antagonist  muscles  to  make  the  system  stable  about  this 
desired  trajectory,  i.e..  the  system  resisted  perturbations  (whether  external  or  due  to  model 
errors)  as  it  moved  along  the  planned  trajectory.  In  our  simulation  of  the  initial  response  to  a 
force  field,  we  assumed  that  the  controller  had  no  knowledge  of  the  forces  in  the  environment, 
i.e..  S  =  0.  Then  we  calculated  the  desired  joint -trajectories.  q~(f).q~(1).<j*(1).  corresponding 
to  straight-line  movements  of  the  hand  towards  the  8  targets.  Finally,  given  the  parameters  in 
Table  1.  we  integrated  Eq.  (4)  for  producing  the  motion  of  the  hand  within  the  test  field. 

The  results  of  this  simulation  are  shown  in  Fig.  7.  We  found  that  there  was  a  striking 
resemblance  between  the  result  of  the  modeled  control  system  (Fig.  7)  and  those  measured  in 
our  subjects  (Fig.  6).  In  particular,  the  presence  of  the  "hooks"  as  well  as  their  orientation 
is  accurately  accounted  for  by  the  modeled  controller.  The  quantitative  differences  between 
model  and  data  are  likely  a  consequence  of  errors  and  simplifications  in  estimating  mechanical 
parameters  of  the  arm  of  each  subject:  for  example,  in  Eq.  12.  we  assumed  a  constant  stiffness 
K  for  the  arm.  This  is  true  when  the  arm  is  near  the  desired  position,  i.e..  when  q  —  q*  is 
small.  However,  it  is  known  that  A  becomes  progressively  and  significantly  smaller  as  the 
distance  between  the  actual  and  desired  hand  positions  increases  (Shadmehr  et  al.  1993).  The 
simulations  also  suffer  from  the  fact  that  our  dynamical  model  neglects  the  small  but  non-zero 
forces  due  to  the  inert  ia  of  the  manipulanduni. 

The  observed  corrective  movements  or  hooks  in  Fig.  G  are  consistent  with  the  operation  of 
a  controller  which  is  attempting  to  move  the  limb  along  a  desired  trajectory  and  bring  it  to  a 
specified  target  position.  Because  this  controller  uses  muscle  viscoelastic  properties  to  define 
an  attractor  region  about  the  desired  trajectory,  the  hand  is  eventually  brought  back  to  near 
the  target  position.  The  hooks  result  from  the  interaction  of  the  viscoelastic  properties  of  the 
muscles  and  the  force  field  which  perturbs  the  system  from  its  desired  trajectory.  Indeed,  the 
results  of  the  model  suggests  that  the  subjects  may  be  executing  a  single  program,  i.e..  that  of 
moving  the  hand  along  a  specified  plan. 


Figure  7:  Simulated  hand  trajectories  for  the  skeletal  dynamics  of  Eq  (11)  with  the  controller  of  Eq. 
(12)  in  the  force  field  of  Fig.  3  before  having  formed  an  internal  model,  i.e..  £  =  0.  Dots  are  10  msec 
apart. 


3.2  Adaptation  to  the  force  field 

After  measuring  the  movements  of  the  arm  in  the  null  field  as  well  as  the  initial  responses  to 
the  unanticipated  force  field,  we  asked  our  subjects  to  keep  executing  reaching  movements  in 
the  force  field.  We  wish  to  stress  that  we  did  not  give  any  instructions  regarding  the  trajectory 
with  which  the  targets  should  have  been  reached.  Nevertheless,  as  the  subjects  practiced  in  the 
force  field,  the  "hooks"  shown  in  Fig.  6  eventually  vanished  and  the  hand  trajectories  became 
increasingly  similar  to  those  observed  in  the  null  field  (Fig.  5).  The  progression  of  hand  position 
traces  as  measured  under  conditions  of  no  visual  feedback  and  in  the  presence  of  the  force  field 
during  the  first,  second,  third,  and  final  target  sets  (each  target  set  consisted  of  250  movements) 
are  shown  in  Fig.  8.  Although  the  force  field  initially  caused  a  significant  divergence  from  the 
trajectory  that  was  normally  observed  for  a  reaching  movement,  with  practice,  the  subjects 
tended  to  converge  upon  this  straight  line  trajectory.  This  recovery  of  the  original  unperturbed 
response  constitutes  a  clear  example  of  an  adaptive  behavior. 

Further  evidence  of  motor  adaptation  is  offered  by  the  significant  change  that  occurred  in 
the  hand  velocity  profile  at  the  onset  of  exposure  to  the  force  field,  and  after  completion  of  the 
practice  trials:  Figure  9A  shows  the  hand  tangential  velocity  traces  obtained  when  the  hand 
was  moving  in  a  null  field  (corresponding  to  the  hand  position  traces  of  Fig.  5).  Consistent 
with  previous  studies  (cf.  Flash  and  Hogan  1985).  these  velocity  traces  are  approximately  along 
straight  lines  and  symmetric  in  time.  The  hand  velocity  traces  at  the  initial  stage  of  practice 
in  the  force  field  (corresponding  to  the  hand  position  traces  of  Fig.  6)  are  shown  in  Fig.  9B. 
After  practice  in  the  force  field,  the  velocity  traces  strongly  resemble  those  recorded  in  a  null 
field:  In  Fig.  9C  we  have  the  velocity  traces  while  the  subject  was  completing  the  practice  trials 
in  the  force  field  (corresponding  to  the  hand  position  traces  of  Fig.  8D).  Although  the  average 
velocity  of  the  hand  trajectory  is  now  larger  (as  compared  to  Fig.  9A).  the  velocity  trace  for 
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Figure  8:  Average  ±  standard-deviation  of  hand  trajectories  during  the  training  period  in  the  force 
field  of  Fig.  3.  Performance  plotted  during  the  first,  second,  third,  and  final  250  targets  (A.  B,  C,  and 
D,  respectively).  All  trajectories  shown  are  under  no-visual  feedback  condition. 


each  target  has  essentially  the  same  pattern  as  that  observed  for  movements  in  a  null  field. 

In  order  to  quantify  the  time  course  of  adaptation,  we  studied  how  the  hand  trajectories 
evolved  as  compared  to  those  observed  in  the  null  field.  For  each  subject,  we  compared  the 
trajectories  in  the  null  field  to  those  obtained  as  the  subject  practiced  in  the  force  field.  This 
comparison  was  made  through  computation  of  a  correlation  coefficient  between  pairs  of  trajec¬ 
tories  (Appendix  I).  We  found  that  the  average  correlation  between  a  trajectory  in  the  null  field 
and  one  in  the  force  field  increased  with  the  amount  of  practice  movements  performed  by  the 
subject  in  the  force  field.  The  computed  correlation  coefficient  for  trajectories  performed  by 
all  subjects  are  shown  in  Fig.  10.  Remarkably,  all  the  subjects  displayed  a  strictly  monotonic 
evolution  of  the  correlation  coefficient.  While  the  initial  value  was  scattered  between  0.5  and 
0.8,  after  500  movements  all  subjects  had  reached  a  similarity  included  between  0.85  and  0.9. 

The  convergence  of  the  trajectories  in  a  force  field  to  that  observed  in  the  null  field  strongly 
suggested  the  existence  of  a  kinematic  plan,  or  desired  trajectory,  independent  of  task  dynamics. 
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Figure  9:  Tangential  hand  velocities  before  and  after  adaptation  to  the  force  field  shown  in  Fig.  3. 

Traces  are.  from  top  to  bottom,  for  targets  at  0°.45o . 31-5°.  A:  Hand  velocities  in  a  null  field  before 

exposure  to  the  force  field  (corresponding  to  position  traces  in  Fig.  5).  B:  Hand  velocities  upon  initial 
exposure  to  the  force  field  (corresponding  to  position  traces  in  Fig.  6).  Hand  velocities  after  1000  reaching 
movements  in  the  field  (corresponding  to  position  traces  in  Fig.  8D). 


Our  subjects  did  not  seem  to  be  aware  of  the  process  of  adaptation  and  of  the  change  in  their 
performance.  The  only  subjective  indication  that  some  adaptive  change  had  occurred  was 
given  by  a  variation  in  the  sense  of  effort  associated  with  the  task.  During  the  first  batch  of 
250  movements  within  the  disturbing  field,  some  subjects  reported  an  intense  sense  of  effort. 
Paradoxically,  this  sense  of  effort  diminished  drastically  after  about  500  movements.  At  the 
end  of  the  training  period  most  subjects  reported  that  they  were  "not  feeling"  the  test  field 
anymore. 

3.3  After-effects 

One  way — although  by  no  means  the  only  way — for  the  subjects  to  recover  the  initial  motor 
performance  (what  we  have  called  the  desired  trajectory)  after  the  exposure  to  the  test  field 
was  by  developing  an  internal  model  of  this  field.  This  internal  model  is  the  term  t  in  the 
expression  of  our  model  controller  (Eq.  12).  On  the  other  hand,  if  after  the  development  of 
an  internal  model  the  test  field  is  removed,  then  one  expects  to  see  a  change  in  the  resulting 
trajectory.  This  change  is  called  an  "after-effect"  of  the  adaptation. 

We  simulated  the  after-effect  by  setting  £  =  Bx  in  our  controller  model  (Eq.  12)  and  E  =  0 
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Figure  10:  The  average  correlation  coefficient  for  movements  in  a  test  force  field  as  compared  to 
movements  in  a  null  field,  as  a  function  of  practice  trials  in  the  force  field.  Each  line  represents  a 
subject. 


in  our  dynamics  model  (Eq.  -I).  This  simulation  corresponds  to  the  assumption  that  subjects 
developed  a  perfect  approximation  of  the  force  field  and  that  this  approximation  remained  in 
effect  as  the  force  field  was  unexpectedly  changed  to  the  null  field.  Again,  the  commanded 
joint-trajectories  corresponded  to  straight-line,  minimum  jerk  movements  of  the  hand  towards 
the  8  targets.  The  results  of  this  simulation  are  shown  in  Figure  11.  Qualitatively,  one  ran  see 
that  the  after  effects  are  ”opposite“  to  the  initial  perturbations  induced  by  the  field  and  shown 
in  Fig.  7.  In  particular.  ( 1)  the  hooks  are  oriented  in  opposite  directions  and  (2)  the  metrics  of 
movements  is  reversed:  long  movements  in  Fig.  7  correspond  to  short  movements  in  Fig.  11  and 
vice  versa.  Indeed  this  qualitative  result  should  be  expected  on  the  basis  of  the  fact  that  the 
differential  equation  which  generates  the  after  effects  can  be  obtained  by  subtracting  the  test 
field  from  the  original  dynamics  within  the  null  field.  Therefore,  these  two  features  described 
above  can  be  regarded  as  a  strong  property,  almost  like  a  "signature",  of  an  internal  model  of 
the  imposed  force  field. 

Experimentally,  we  tested  the  hypothesis  that  adaptation  in  the  subjects  involved  develop¬ 
ment  of  an  internal  model  by  removing  the  force  field  at  the  ons<  of  movement  and  recording 
the  after-effect.  We  removed  the  field  at  random  trials  during  the  period  of  adaptation  to  the 
test  field.  We  found  that  the  magnitude  of  the  after-effects  grew  with  the  length  of  exposure 
to  the  force  field:  Figure  12  illustrates  the  temporal  progression  of  after-effects,  as  measured 
under  conditions  of  no  visual  feedback  and  in  the  null  field,  during  the  first,  second,  third, 
and  final  target  sets.  The  size  of  the  after-effect,  as  indicated  by  the  deviation  of  the  hand 
trajectory  from  a  straight  line,  grew  with  practice  in  the  force  field.  By  the  final  target  set  (Fig. 
12D).  the  hand  trajectory  in  the  null  field  was  significantly  skewed.  Remarkably,  the  observed 
after  effects  at  the  end  of  the  adaptation  period  had  the  same  qualitative  features  as  those 
predicted  by  our  simulation  of  an  internal  model  within  the  null  field  (Fig.  11).  In  particular, 
by  comparing  Fig.  8  with  Figure  121).  one  can  see  that  ( 1 )  all  the  hooks  had  reversed  directions 
and  (2)  the  metrics  of  movement  has  changed  as  in  the  simulation. 

This  finding  is  consistent  with  the  hypothesis  that  subjects  adapted  to  the  force  field  by 
creating  an  internal  model  that  approximated  the  dynamics  of  the  environment.  In  addition, 
the  data  shown  in  Fig.  12  indicate  that  most  of  the  development  of  this  internal  model  took 
place  during  practice  trials  in  the  first  and  second  target  sets,  as  indicated  by  the  progressive 
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Figure  11:  Simulated  after-effect  trajectories:  Hand  trajectories  for  the  skeletal  dynamics  of  Eq.  (11) 
in  a  null  force  field  with  the  controller  of  Eq.  (12).  assuming  that  the  controller  had  formed  an  internal 
model  of  the  force  field  shown  in  Fig.  3. 


development  of  after-effects  during  these  periods  (Fig.  12A  and  12B).  From  this  observation 
one  would  expect  that  performances  of  the  subjects  in  the  force  field  should  have  shown  most 
of  its  improvement  rather  early  in  the  training.  This  is  in  agreement  with  the  correlation  curves 
shown  in  Fig.  10:  in  general,  for  all  subjects  the  correlation  coefficient  increased  most  rapidly 
at  the  early  stage  of  exposure  to  the  field,  indicating  that  the  subjects  had  composed  a  fairly 
accurate  internal  model  of  the  imposed  force  field  by  the  midpoint  of  the  training  session. 

3.4  Transferred  after-effects 

Our  results  in  Fig.  (12)  provide  a  strong  indication  that  adaptation  occurred  through  the 
development  of  an  internal  model  of  the  applied  mechanical  environment.  What  is  the  structure 
of  this  model  and  how  is  it  represented  in  the  nervous  system?  A  priori,  there  are  several 
legitimate  hypotheses.  This  internal  model  can  be  regarded  as  a  mapping  between  the  state  of 
the  arm  (position  and  velocity)  and  the  corresponding  force  exerted  by  the  environment.  In  an 
artificial  system,  one  may  implement  such  a  mapping  as  a  look-up  table  by  storing  away  in  some 
memory  location  the  forces  encountered  at  each  state  visited  during  the  period  of  adaptation 
(cf.  Raibert.  1978.  Miller  1978.  Atkeson  and  Reinkensmever  1989).  This  type  of  local  mapping 
has  also  been  proposed  in  biological  models,  such  as  the  one  formulated  by  Albus  ( 1975)  for  the 
cerebellum.  In  psychophysics,  this  kind  of  model  is  called  a  "specific  exemplar  model"  and  has 
been  used  to  suggest  a  mechanism  for  learning  a  motor  task  (cf.  Chamberlin  and  Magill  1992). 
Of  course,  if  the  internal  model  were  a  look-up  table,  adaptation  would  occur  only  at  (or  in 
the  neighborhood  of)  the  visited  states.  As  a  consequence,  no  after  effect  should  be  detectable 
if.  after  the  adaptation,  the  null  field  was  presented  at  some  location  outside  the  neighborhood 
visited  during  the  training  period. 


Figure  12:  After  efTects  of  adaptation  to  the  force  field  shown  in  Fig.  3  at  t he  right  workspace.  Shown 
are  average  ±  standard-deviation  of  the  hand  trajectories  while  moving  in  a  null  field  during  the  training 
period  for  the  first,  second,  third,  and  final  '2-r)0  targets  (A.  B.  C.  and  D.  respectively).  All  trajectories 
shown  are  under  no- visual  feedback  condition. 


To  test  t his  hypothesis  regarding  the  representation  of  the  internal  model  as  a  local  associ¬ 
ation  between  states  and  forces,  we  asked  our  subjects  to  make  reaching  movements  in  the  null 
field  at  the  left  workspace  before  and  after  having  been  exposed  to  the  test  field  at  the  right 
workspace  (workspaces  are  shown  in  Fig.  2).  Fig.  13A  shows  a  set  of  trajectories  in  the  null  field 
at  the  left  workspace.  These  trajectories  were  obtained  before  the  subject  practiced  movements 
in  the  force  field  at  the  right  workspace.  Figure  13B  shows  the  average  trajectories  obtained 
from  the  same  subject,  in  the  same  left  workspace  and  with  the  same  null  field,  but  after  the 
subject  had  adapted  to  the  field  in  the  right  workspace.  Clearly,  there  were  substantial  after¬ 
effects  in  the  left  workspace  resulting  from  adaptation  in  the  right  workspace.  This  finding  is 
not  compatible  with  the  hypothesis  that  subjects  developed  an  internal  model  by  building  a 
look-up  table,  that  is.  a  local  association  between  visited  states  and  experienced  forces.  On 
the  contrary,  the  internal  model  appeared  to  extend  and  "generalize''  quite  broadly  outside  the 
portion  of  workspace  explored  during  the  period  of  adaptation.  This  pattern  of  generalization, 
as  evidenced  by  the  transferred  after-effects,  was  similar  in  all  subjects,  regardless  of  whether 
they  had  trained  at  the  right  workspace  in  an  endpoint  translation  invariant  field  (Eq.  1 )  or  a 
joint  translation  invariant  field  (Eq.  3). 
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Once  we  had  established  that  the  internal  model  was  not  merely  a  local  association  between 
states  and  forces,  a  question  that  remained  was  how  the  internal  model  extrapolated  outside  the 
region  where  the  subject  had  trained.  Concerning  this  issue,  one  may  define  two  broad  classes 
of  generalizations.  In  one  class,  the  generalization  may  be  the  outcome  of  some  cognitive 
inference  about  the  mechanical  properties  of  the  environment.  1  or  example,  if  we  are  stirring 
a  can  of  paint,  from  physics  we  know  that  we  should  experience  the  same  forces  on  our  hand 
(for  a  given  hand  trajectory)  regardless  of  the  location  of  the  paint  can  in  the  workspace  of  our 
arm.  In  this  sense,  we  would  expect  the  viscous  field  representing  the  mechanical  properties 
of  the  paint  to  be  translation  invariant  in  endpoint  coordinates.  This  expectation  would  be 
reflected  in  the  geometric  structure  of  our  internal  model:  the  internal  model  would  be  a  map 
between  motion  and  forces  in  an  extrinsic  coordinates.  Consistent  with  the  properties  of  the 
environment,  it  would  predict  identical  forces  acting  on  the  hand  when  movements  are  done  in 
the  novel  region  of  the  workspace  (as  compared  to  movements  in  the  region  where  we  trained). 
As  a  consequence,  the  adaptation  to  a  velocity-dependent  field  in  the  right  workspace  would 
also  imply  the  adaptation  to  the  same  force  field  in  the  left  workspace.  In  order  to  achieve  this 
type  of  generalization,  it  is  necessary  to  postulate  the  existence  of  further  computations  that 
transform  predicted  end-point  forces  (output  of  the  internal  model)  into  muscle  torques  across 
the  workspace. 

Alternatively,  adaptation  may  be  through  composition  of  an  internal  model  that  does  not 
require  further  coordinate  transformations:  it  simply  represents  the  environment  in  terms  of  a 
map  between  motion  and  forces  in  the  coordinate  system  of  its  sensors  and  actuators.  This 
model  would  be  implemented  by  a  controller  that,  during  execution  of  the  task,  effectively 
changes  the  dynamical  behavior  of  the  muscles  to  approximate  and  compensate  for  the  force 
field  in  the  region  visited  during  adaptation.  Indeed,  these  changes  in  muscle  behavior  are 
bound  to  have  some  effect  beyond  the  region  in  which  the  subject  was  trained.  For  example, 
by  activating  the  pectoralis  (a  single-joint  flexor  of  the  shoulder)  a  subject  may  compensate  a 
force  directed  toward  the  elbow.  Let  us  assume  that  this  muscle  activation  is  the  outcome  of 
the  internal  model  and  that  there  is  no  other  computation  dedicated  to  perform  any  coordinate 
transformation.  Then,  when  the  arm  is  at  another  configuration  the  internal  model  will  produce 
the  same  muscle  activation  if  the  same  state  of  the  arm  is  given  as  input.  Of  course  this  time  the 
output  force  will  be  different  because  there  is  a  different  geometrical  relation  between  pectoralis 
activation  and  endpoint  force  in  the  new  location  of  the  arm.  In  this  second  case,  we  would 
expect  the  internal  model  to  be  translation  invariant  in  an  intrinsic  coordinate  system,  and 
generalization  to  be  a  side-effect  of  biomechanics. 

Our  experimental  results  clearly  favor  this  second  scenario  where  the  forces  in  the  environ¬ 
ment  are  generalized  in  terms  of  an  intrinsic  coordinate  system,  i.e..  in  terms  of  torques  on 
joints.  The  after-effects  observed  at  the  left  workspace  (Fig.  13B)  were  significantly  different 
than  those  observed  at  the  right  (Fig.  1'2D).  For  example,  compare  movements  to  targets  at 
45°,  135°,  225°  and  315°  in  each  figure.  These  differences  suggested  that  based  on  the  internal 
model  formed  after  practice  in  the  right  workspace,  the  subjects  expected  to  interact  with  very 
different  forces  at  the  left  workspace.  We  tested  this  hypotheses  directly  by  having  subjects 
which  practiced  in  the  field  shown  in  Fig.  3A  at  the  right  workspace,  make  movements  without 
visual  feedback  in  the  field  shown  in  Fig.  4 A  at  the  left  workspace.  The  results  are  shown 
for  a  typical  subject  in  Fig.  14A:  This  subject  belonged  to  Croup  1.  i.e..  trained  at  the  right 
workspace  on  the  endpoint  translation  invariant  field  described  by  Eq.  (1).  Although  forces  in 
Figs.  3A  and  4A  are  nearly  orthogonal,  the  subject  performed  near  perfectly  (p  =  0.91 )  at  the 
left  workspace  in  the  field  of  Fig.  4 A.  The  same  subject's  performance  in  the  left  workspace 
was  poor  {p  =  0.02)  in  the  field  of  Fig.  3A  (shown  in  Fig.  14B).  This  indicated  that  the  subject 
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Figure  13:  Transferred  after-effects:  Average  ±  standard-deviation  of  hand  trajectories  while  moving 
in  a  null  field  at  the  left  workspace.  A:  Before  the  subject  practiced  movements  in  the  field  of  Fig.  3 
at  the  right  workspace.  B:  After  the  subject  practiced  movements  in  the  field  of  Fig.  3  at  the  right 
workspace. 


generalized  the  force  field  in  terms  of  an  intrinsic  coordinate  system. 

The  performance  of  all  subjects  in  the  two  force  fields  at  the  left  workspace  was  quantified 
by  computing  the  correlation  coefficient  between  the  trajectories  in  each  force  field  and  the 
trajectory  in  the  null  field.  These  coefficients  are  shown  in  Fig.  15.  The  results  consistently 
indicated  that  subjects  retained  the  kinematic  features  of  the  adapted  behavior  when  the  envi¬ 
ronment  was  translated  to  the  novel  region  of  the  workspace  in  joint  coordinates,  and  not  when 
this  translation  was  in  endpoint  coordinates.  This  rejected  the  hypothesis  that  the  internal 
model  attributed  a  hand-based  invariance  to  the  environmental  field. 


4  Discussion 

We  used  the  paradigm  of  a  programmable  mechanical  environment  in  order  to  study  how  the 
motor  control  system  adapts  to  a  change  in  the  dynamics  of  a  well  rehearsed  task.  The  task 
which  we  considered  was  a  reaching  movement  wdiere  the  hand  interacted  with  a  force  field 
produced  by  a  robot  manipulandum.  We  chose  a  force  field  which  significantly  changed  the 
dynamics  of  the  task,  resulting  in  a  large  change  in  the  trajectory  that  the  hand  took  in  making 
a  reaching  movement  (as  compared  to  moving  in  a  null  field).  The  objective  was  to  observe 
how  the  subjects  responded  to  this  change  in  the  system  dynamics. 

We  tested  the  hypothesis  that  in  programming  a  reaching  movement,  the  (’NS  initially 
specifies  a  desired  trajectory  of  the  hand  and  then  uses  an  internal  model  of  the  limb's  dynamics 
to  produce  torques  appropriate  for  moving  the  hand  along  this  desired  trajectory.  When  the 
limb's  dynamics  were  changed  (by  imposing  a  force  field  on  the  hand),  the  internal  model  was 
no  longer  accurate,  resulting  in  the  hand  moving  along  a  trajectory  that  deviated  from  the 
desired  behavior.  This  error,  as  observed  over  repeated  movements,  led  to  gradual  updating  of 
the  internal  model  so  that  it  eventually  approximated  the  new  dynamics  of  the  limb.  In  the 
current  study,  we  found  evidence  regarding  existence  of  a  desired  trajectory  and  that  the  motor 
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Figure  14:  Average  ±  standard-deviation  of  hand  trajectories  during  initial  exposure  to  a  field  at  the 
left  workspace  immediately  after  the  subject  practiced  movements  in  the  field  shown  in  Fig.  at  the 
right  workspace.  A:  Performance  at  the  left  workspace  in  the  field  of  Fig.  -F  B:  Performance  at  the  left 
workspace  in  the  field  of  Fig.  d. 


controller  achieved  this  desired  performance  via  an  explicit  composition  of  an  internal  model. 

4.1  Evidence  for  a  desired  trajectory 

The  task  of  moving  the  hand  to  a  target  position  is  ill-posed  in  the  sense  that  the  subject 
may  choose  from  an  infinite  set  of  trajectories  to  achieve  the  goal.  Vet.  for  two-dimensional 
movements  with  moderate  accuracy  requirements  (such  as  our  task),  it  has  been  demonstrated 
that  subjects  tend  to  move  their  hand  smoothly  and  along  a  straight  line  (Morasso  19X1. 
Soechtingand  Lacquaniti  1981.  Flash  and  Hogan  1985).  Reaching  movements  are  characterized 
by  fairly  constant  duration,  whatever  their  direction  or  extent,  and  a  bell-shaped  curve  of  the 
tangential  hand  velocity  versus  time  (Morasso  1981).  Here  we  confirmed  this  observed  this 
phenomenon  as  subjects  performed  the  task  in  a  null  field  (Figs,  (i  and  9A).  In  addition,  we 
found  that  when  the  dynamics  of  the  task  were  changed  by  imposing  a  force  field  onto  the  hand, 
the  result  was  hand  trajectories  which  deviated  significantly  from  this  smooth,  straight  line  path, 
as  is  shown  in  the  position  traces  of  Fig.  7.  and  velocity  traces  of  Fig.  9B.  Nevertheless,  through 
practice,  the  subjects'  hand  trajectories  converged  to  the  trajectory  observed  during  null  field 
conditions  (Figs.  8  and  10).  This  convergence  was  gradual  but  monotonic  in  all  subjects, 
consistent  with  an  adaptive  process  whose  goal  was  to  compensate  for  the  forces  imposed  by 
the  field  and  return  the  hand's  trajectory  to  that  produced  before  the  perturbation.  This  finding 
suggests  that  the  kinematics  observed  in  reaching  movements  is  not  merely  a  consequence  of 
arm  dynamics  but  reflects  the  presence  of  a  plan.  i.e..  a  desired  trajectory. 

4.2  Properties  of  the  desired  trajectory 

Can  we  specify  the  criterion  that  the  CNS  uses  to  specify  the  desired  trajectory?  The  desired 
performance  of  a  controlled  system  is  usually  established  by  a  criterion,  or  optimization  prin¬ 
ciple.  expressed  in  a  particular  coordinate  system  (e.g..  the  coordinate  system  of  the  task.  cf. 
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Figure  15:  Summary  of  performance  in  the  left  workspace  after  training  at  the  right  workspace.  Sub¬ 
jects  in  Group  1  trained  on  the  field  given  by  Eq.  (I).  while  subjects  in  Group  2  trained  in  field  given  by 
Eq.  (3).  The  two  fields  were  essentially  identical  in  the  right  workspace  but  orthogonal  at  the  left.  Shown 
are  average  correlation  coefficients  for  movements  in  the  left  workspace  in  a  force  field  as  compared  to 
movements  in  a  null  field  for  the  same  subject.  Light  gray  bars  are  for  movements  in  field  given  by 
Eq.  (3)  while  dark  gray  bars  are  for  movements  in  field  given  by  Eq.  ( 1 ).  Performance  was  significantly 
better  in  both  Groups  when  the  force  field  was  transferred  to  the  left  workspace  in  terms  of  joint  torques 
rather  than  end  point  forces. 

Flash  and  Hogan  1985.  Jordan  and  Rumelhart  1992.  Jordan  1993).  For  skilled  movements  of 
the  arm.  this  criterion  appears  to  be  one  of  smoothness.  Specifically,  in  the  context  of  reaching 
movements  in  the  horizontal  plane.  Flash  and  Hogan  (1985)  have  noted  that  the  hand's  tra¬ 
jectory  is  well  described  by  a  function  which  maximizes  a  measure  of  smoothness.  In  a  similar 
work.  Stein  et  al.  (1988)  have  shown  that  in  the  single  joint  case,  the  optimal  fit  to  joint  veloc¬ 
ity  is  a  Gaussian  function  which  is  also  consistent  with  an  optimization  of  smoolhness  ( Poggio 
and  Girosi  1990).  Even  in  more  complicated  tasks  such  as  reaching  around  obstacles,  there  is 
evidence  that  with  practice,  the  trajectory  of  the  hand  becomes  progressively  smoother  (Abend 
et  al.  1982.  Schneider  and  Zernicke  19PP).  Therefore,  this  optimization  of  smoothness  in  terms 
of  the  trajectory  of  the  hand  serves  as  a  possible  computational  principle  that  the  ('NS  might 
be  using  to  describe  the  desired  performance  during  a  reaching  movement. 

A  characteristic  of  the  above  hypothesis  is  that  the  desired  behavior  of  the  arm  is  achieved 
via  a  purely  kinematic  principle,  i.e..  smoothness  of  the  change  in  the  position  of  the  hand.  This 
is  appealing  as  it  would  imply  a  separation  between  the  planning  and  the  execution  stages  of  the 
motor  task:  as  long  as  the  task  is  to  move  the  hand  to  a  target  position,  the  desired  trajectory 
remains  a  smooth,  straight  line  path  (in  task  coordinates),  regardless  of  whether  a  force  field 
is  present.  As  Bernstein  (1967)  noted,  this  kind  of  separation  of  planning  from  execution  is 
inherent  to  a  hierarchical  structure  where  a  change  in  the  dynamics  of  the  controlled  system 
does  not  affect  the  definition  of  the  desired  behavior. 

Alternatively,  one  can  forward  other  computational  principles  which  the  CNS  might  be 
using  to  define  a  desired  trajectory  where  the  stage  of  planning  is  highly  dependent  on  the 
stage  of  execution.  For  example,  consider  that  the  CNS  could  specify  a  des>red  trajectory  for 
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ill*1  hand  such  that  the  (argot  is  reached  the  most  "effortlessly”,  when*  an  effort  is  defined  as 
a  measure  of  energy,  based  on  the  physical  cost  of  the  movement  (Nelson  19s:{).  or  based  on 
changes  in  the  forces  or  torques  on  the  joints  (Fno  el  al.  19X9).  In  fact,  it  has  been  shown 
that  the  smoothness  and  straight  line  properties  of  the  hand  trajectory  may  be  a  by-product 
of  a  minimum  lorque-ch  mge  criterion  ( (  no  et  al.  19X9).  However,  in  contrast  to  the  previous 
approach,  based  on  the  this  scenario  the  desired  trajectory  would  change  as  a  function  of  the 
dynamics  of  the  task,  closely  linking  the  process  of  planning  to  that  of  execution. 

The  force  fields  that  we  imposed  on  the  hand  as  a  reaching  movement  was  attempted  changed 
the  dynamics  of  the  arm  quite  drastically  (cf.  Fig.  (j).  Nevertheless,  through  practice,  the  sub¬ 
jects’  hand  trajectories  converged  upon  the  trajectory  observed  during  unperturbed  conditions, 
as  shown  in  Fig.  X.  and  quantified  in  Fig.  10.  The  only  major  difference  was  an  increase  in 
peak  velocity  (on  average',  peak  velocity  increased  by  {Z  19  with  respect  to  movements  in  a  null 
field,  cf.  Fig.  DC),  a  phenomenon  which  has  been  linked  to  repetition  of  a  motor  task  by  other 
investigators  (cf.  Kerr  1992).  This  observed  convergence  to  the  unperturbed  trajectory  argues 
for  an  explicit  description  of  a  desired  trajectory  w  hose  kinematics  are  essentially  independent 
of  tin  dynamics  of  the  task,  in  line  with  the  notion  of  a  separation  of  the  planning  from  an 
execution  stage. 

Recent  results  from  Flash  and  Gurevich  (1992)  have  provided  evidence  supporting  this 
idea  of  an  invariant  kinematic  plan  for  a  task  where  the  hand  int.'racted  with  a  static  load. 
Similarly.  Farquaniti  et  al.  (19X2)  found  that  subjects  which  were  asked  to  move  a  2.5  kg 
weight  did  so.  after  some  practice  trials,  along  essentially  the  same  trajectory  as  when  moving 
the  hand  without  the  weight.  Our  work  has  shown  that  even  when  the  change  in  the  dynamics 
of  the  limb  is  fairly  severe,  the  response  is  a  convergence  to  the  trajectory  observed  before  the 
change,  albeit  this  convergence  may  take  place  over  a  fairly  long  practice  period  (500  to  1000 
movements  in  our  case,  as  shown  in  Fig.  10).  This  is  similar  to  the  conclusion  reached  for  single 
degree  of  freedom  movements  by  Ruitenbeek  (  19X4).  who  found  that  when  a  subject  interacted 
with  a  manipulandum  with  variable  dynamics,  practice  led  to  a  trajectory  that  was  invariant 
with  respect  to  the  dynamics  of  the  manipulandum.  These  results  are  not  compatible  with  the 
idea  that  the  process  of  planning  is  mainly  influenced  by  the  dynamics  of  the  task,  as  one  would 
expect  different  desired  trajectories  for  different  environments  since  a  change  in  the  environment 
causes  a  change  in  the  system’s  dynamics.  Indeed,  invariance  of  the  plan  with  respect  to  the 
dynamics  suggests  that  there  may  be  specific  elements  in  the  motor  control  hierarchy  which  are 
concerned  with  the  description  of  a  desired  behavior  of  a  limb  in  terms  of  pure  kinematics. 

4.3  Adaptation  through  composition  of  an  internal  model 

Convergence  of  the  hand  trajectories  while  interacting  with  the  novel  force  field  is  an  indica¬ 
tion  of  the  adaptation  of  the  motor  controller.  We  hypothesized  that  this  adaptation  was  via 
composition  of  an  internal  model  of  the  imposed  force  field,  in  this  scenario,  the  internal  model 
is  a  mechanism  by  which  the  nervous  system  predicts  the  forces  that  would  be  acting  on  the 
hand  as  it  performs  the  task. 

The  force  field  which  was  imposed  on  the  hand  had  the  property  of  being  dependent  on 
the  velocity  of  the  hand,  resulting  in  a  situation  where  the  subject  did  not  know  whether  the 
field  was  “on”  or  "off"'  until  the  movement  was  actually  initiated.  However,  during  the  training 
period,  in  91(/<  of  the  movements  the  field  was  on.  presumably  facilitating  formation  of  a  model 
of  the  force  field  which  the  (.’NS  might  use  as  a  part  of  a  control  system  to  move  the  hand  along 
the  desired  trajectory  (for  the  remaining  movements  the  field  was  off  in  order  to  measure  any 
after-effects  of  adaptation).  We  suggested  that  this  control  system  may  be  represented  as  the 


21 


sum  of  three  components:  an  internal  motU'l  describing  t lit'  dynamics  of  tin*  skeletal  system  of 
the  arm  when  moving  in  a  null  field,  an  internal  model  describing  the  dynamics  of  the  force 
field  imposed  on  the  hand,  and  a  viscoelastic  or  feedback  system  intended  to  stabilize  the  arm 
about  the  desired  tra  jectory  in  case  of  errors  in  these  models. 

Initially,  the  subject  had  not  formed  a  model  of  the  force  field,  resulting  in  a  discrepancy 
between  the  expected  dynamics  of  the  arm  and  the  dynamics  actually  present.  This  "model 
error"  led  to  hand  trajectories  (  Fig.  7)  which  were  significantly  different  than  the  desired  trajec¬ 
tories.  Indeed,  we  found  excellent  correspondence  between  trajectories  produced  by  the  model 
controller  (Fig.  7)  and  those  observed  in  the  movements  of  the  subjects  (Fig.  (i).  In  particu¬ 
lar.  we  observed  that  the  responses  to  the  sudden  presentation  of  the  field  were  characterized 
by  a  sharply  curved  trajectory  that  we  described  as  a  "hook".  A  possible  interpretation  for 
this  hook  would  be  that  the  hand  starts  the  movement  along  a  wrong  direction  and  that  the 
resulting  error  is  corrected  by  a  second  movement.  However,  there  is  a  simpler  interpretation 
of  the  same  result  that  does  not  make  appeal  to  explicit  correction  processes.  According  to 
this  interpretation,  the  corrective  movement  is  a  by-product  of  the  interaction  between  the 
mechanical  properties  of  the  arm  as  imposed  by  the  controller  (the  function  S  in  Eq.  9)  and  the 
force  field  imposed  on  the  hand.  Presence  of  the  hook  as  well  as  the  initial  error  in  movement 
direction  are  systematically  predicted  by  the  simulation  of  the  dynamics  of  this  model.  We 
wish  to  emphasize  that  our  reason  to  favor  this  hypothesis,  at  present,  rests  uniquely  on  its 
computational  simplicity  as  compared  to  the  hypotheses  which  requires  an  explicit  correction 
process. 

If  the  adaptive  process  was  via  composition  of  an  internal  model  of  the  imposed  force  field, 
then  we  argued  that  bv  removing  the  field,  once  again  there  should  be  a  discrepancy  between 
expected  and  actual  dynamics  of  the  system.  To  test  this,  in  9'7f  of  the  movements  during  the 
training  period  the  force  field  was  returned  to  its  null  condition.  Our  simulation  of  the  adaptive 
controller  suggested  that  there  would  be  aftc of  adaptation  (Fig.  11).  We  found  that 
when  the  field  was  unexpectedly  removed,  the  subjects  produced  trajectories  similar  to  those 
predicted  by  the  simulation.  The  "magnitude"  of  the  observed  after-effects  increased  gradually 
with  the  practice  period  ( Fig.  12).  This  progressive  buildup  of  after-effects  was  further  evidence 
that  the  C’NS  improved  performance  via  an  explicit  composition  of  an  internal  model. 

Of  course,  it  is  easy  to  envision  a  system  whose  performance  in  response  to  a  perturbation 
improves  not  because  of  an  internal  model,  but  due  to  an  increase  in  the  stiffness  of  the  system 
about  the  desired  trajectory.  This  alternative  strategy  may  be  achieved  by  an  increase  in  the 
coactivation  of  the  muscles.  As  a  consequence,  movements  would  become  more  insensitive  to 
changes  in  the  external  forces.  It  is  easy  to  show  that  modest  increases  in  arm  stiffness  (about 
3  folds  with  respect  to  the  values  measured  in  posture)  leads  to  almost  perfect  performance 
in  the  force  field.  However,  if  this  strategy  is  chosen  as  the  mode  of  adaptation,  then  it  is 
evident  that  exposure  to  a  force  field  would  not  cause  an  after-effect  in  a  null  field.  Indeed, 
the  fact  that  practice  does  cause  progressively  larger  after-effects  (Fig.  12)  is  strong  evidence 
against  the  hypothesis  that  the  convergence  of  trajectories  is  due  to  a  mechanism  such  as  global 
coactivation  of  muscles.  This  is  in  agreement  with  measurements  of  van  Emmerik  ( 1991 )  where 
it  was  shown  that  during  learning  of  a  novel  arm  movement  the  stiffness  of  the  limb  decrease  i s 
slightly. 

4.4  Transfer  properties  of  the  internal  model 

As  a  subject  practiced  reaching  movements  in  a  force  field,  the  after-effects  suggested  that  an 
internal  model  of  that  field  was  being  composed.  We  can  represent  the  information  contained 


in  this  internal  model  as  a  map  whose  input  is  the  state  of  the  arm  and  whose  output  is  a  force. 
Indeed,  this  output  is  the  force  predicted  (tv  the  internal  model  that  would  he  imposed  by 
the  environment  as  the  arm  passed  through  a  given  state.  Therefore,  the  internal  model  was  a 
map  which  approximated  the  force  field  imposed  by  the  mechanical  environment,  anil  the  task 
for  the  subject  was  to  learn  to  perform  this  approximation  from  a  set  of  examples  (where  the 
examples  were  provided  as  the  subject  made  arm  movements  in  the  force  field).  What  are  the 
mechanisms  underlying  this  approximation?  The  after  effects  can  again  serve  as  a  powerful 
tool  to  reveal  some  of  the  properties  of  the  internal  model. 

The  description  of  a  biological  learning  task  can  often  be  represented  as  approximation  of 
a  sensorimotor  map.  This  approach  has  recently  been  implemented  by  a  distributed  technique 
inspired  by  the  architecture  of  the  nervous  system.  In  this  approach,  the  mapping  is  formed  via 
interaction  of  a  set  of  non  linear  computational  elements  which  represent  neuron  like  structures 
(cf.  Barto  1 98.9.  Poggio  1990).  For  example,  for  the  task  of  motor  learning,  combinations  of 
non-linear  basis  functions  have  been  used  to  implement  an  internal  model  which  represents 
the  inverse  dynamics  of  a  multi-joint  limb  (Raibert  and  Wimberly  1 9X1.  Kawato  19X9.  Jordan 
1990.  Shadmehr  1990.  Kawato  and  Gomi  1992.  Jordan  1993).  mapping  states  of  the  limb  to 
forces  (e.g.,  Eq.  0).  These  results  have  provided  an  algorithm  by  which  an  internal  model 
may  be  constructed  by  an  intelligent  system.  However,  little  has  been  learned  regarding  the 
properties  of  the  computational  elements  with  which  the  nervous  system  might  be  performing 
this  adaptive  process. 

Consider  that  a  property  of  the  computational  elements  (e.g..  basis  functions  or  “neurons" 
in  a  neural  network)  used  in  learning  a  map  is  their  spatial  bandwidth,  i.e..  the  size  of  their  sup¬ 
port  or  “receptive  field"  in  the  input  space.2  The  size  and  location  of  the  receptive  fields  greatly 
influence  how  the  learning  system  interpolates  between  states  which  it  has  visited  during  train¬ 
ing.  and  whether  it  can  generalize  to  regions  beyond  the  boundary  of  its  training  data  (Poggio 
and  Girosi  1990).  In  fact,  research  in  visual  perception  has  used  the  notion  of  generalization 
to  make  an  inference  regarding  the  receptive  fields  of  the  computational  elements  used  by  the 
visual  system  to  learn  a  map:  in  a  hyperacuity  task.  Poggio  et  al.  (1992)  have  shown  that  if 
the  computational  elements  have  narrow  receptive  fields  similar  to  those  found  in  components 
of  early  vision,  a  subject  should  not  be  able  to  generalize  to  tasks  which  are  slightly  different 
than  those  on  which  the  subject  had  been  trained  on — a  prediction  which  agrees  with  results  of 
experiments  (Poggio  et  al.  1992).  Simply  said,  this  is  because  during  the  learning  of  the  task, 
only  the  “weights"  of  those  elements  which  are  activated  by  the  input  are  changed,  and  if  these 
elements  respond  to  only  a  narrow  range  of  inputs,  then  the  system  can  not  generalize  to  a 
region  outside  the  training  data.  Computation  emerges  from  the  superposition  of  the  receptive 
fields  of  the  activated  elements.  The  implication  is  that  for  some  visual  recognition  tasks,  the 
nervous  system  approximates  a  map  in  terms  of  these  "low-level"  elements  which  encode  visual 
information  using  fairly  narrow  receptive  fields  (Poggio  1990). 

In  a  moto"  task,  the  equivalent  low-level  elements  are  muscles  and  their  associated  spinal 
(Bizzi  et  al.  1991)  and  supra-spinal  (Berthieret  al.  1993)  neural  control  pathways.  For  example. 
Giszter  et  al.  (1993)  have  quantified  behavior  of  some  of  the  neural  circuits  and  the  associated 
muscles  in  a  frog's  spinal  cord:  each  circuit  is  a  collection  of  interneurons  connected  to  a  group  of 
motor  units.  When  a  circuit  is  activated  through  microstimulation,  the  muscles  generate  a  time 
varying  force.  This  force  depends  on  the  configuration  of  the  limb  and  may  be  represented  as  a 
force  field,  e.g..  an  endpoint  force  as  a  function  of  the  position  of  the  tip  of  the  limb.  Therefore. 

"’The  support  of  a  function  is  that  region  of  the  function's  domain  over  which  the  output  value  is  different 
from  zero.  In  neurophysiology,  the  support  of  a  function  which  represents  the  input  -output  behavior  of  a  cell  is 
often  referred  to  as  a  “receptive  held". 
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computationally  the  behavior  of  the  low  level  elements  in  tin*  motor  control  hierarchy  is  to 
produce  an  output  force  as  a  function  of  the  input  activation  to  the  spinal  neural  circuitry  and 
the  position  of  the  limh  and  time  (Mussa  Ivaldi  et  al.  1990). 

In  a  general  framework,  it  seems  more  plausible  to  assume  that  the  pattern  of  forces  gen¬ 
erated  by  such  a  spinal  controller  depends  upon  velocity  of  the  limb  as  well  as  its  position. 
Therefore,  we  can  describe  the  output  force  for  a  given  amount  of  activation  of  a  spinal  cir¬ 
cuitry  as  a  time  varying  vector  valued  function  o(q.  q. /jr).  where  c  is  an  activation  parameter. 
This  time  varying  force  field  is  essentially  a  wave  expressing  the  input  output  behavior  of  a 
motor  computational  <  h  no  nl  within  the  central  nervous  system.  In  theory,  a  collection  of  these 
computational  elements  can  be  used  in  a  motor  learning  task:  A  finding  of  the  spinal  micros¬ 
timulation  experiments  (Bizzi  et  al.  1991.  Giszter  et  al.  1983)  has  been  that  the  simultaneous 
stimulation  of  two  separate  sites  resulted  in  the  summation  of  the  fields  obtained  from  the  sep¬ 
arate  stimulation  of  each  site.  In  other  words,  the  output  of  the  motor  computational  elements 
add  when  two  are  activated.  Based  on  this  property  of  superposition,  a  simple  framework  for 
motor  learning  in  terms  of  these  computational  elements  can  be  proposed:  Consider  the  force 
field  which  results  when  all  the  motor  computational  elements  are  simultaneously  activated: 
Y,  o,(q.  q.  /|c().  This  expression  assumes  a  particularly  tractable  form  if  the  control  parameters 
of  the  computational  elements  can  be  considered  as  linear  scaling  coefficients:  Y  ,  <'iO,(q.  q.  t ). 
In  this  context,  the  problem  of  producing  a  desired  motor  behavior  is  equivalent  to  that  of 
approximating  the  field  described  by  the  controller  in  Eq.  (8).  i.e..  solve  for  c,  such  that: 

Y^CiOt(q,q.t)  ~  V(q.q.t)-  S(q  -  q’(t).q  -  </“(/)) 

i 

In  the  above  we  have  the  controller  for  arm  movements  implemented  via  a  set  of  motor  compu¬ 
tational  elements,  svhere  each  element  is  a  time-varying  force  field,  or  wave.  Mathematically, 
the  idea  is  to  think  of  the  muscles  and  their  spinal  circuitry  as  a  computational  elemenl  in  a 
motor  approximation  task  ( Mussa- Ivaldi  1992.  Mussa-Ivaldi  and  Giszter  1992).  In  relation  to 
other  theories  in  motor  learning,  each  computational  elements  can  be  thought  of  as  a  primitive 
movement,  or  motor  schema  ( Arbib  1985). 

This  idea  of  implementing  a  controller  via  a  distributed  network  of  motor  computational 
elements  may  be  easily  extended  to  the  task  of  building  an  internal  model  of  an  environmental 
force  field:  given  a  desired  field  t'(q.q.t).  the  adaptive  controller  must  find  a  set  of  control 
parameters,  e,  such  that: 

^(iOdq.q.t)  ~  f(q.q.t) 

l 

Using  this  notation,  our  proposal  consists  in  separating  an  invariant  "task  representation",  i.e.. 
a  set  of  coefficients,  c,.  from  an  "environment  representation",  i.e..  a  set  of  coefficients.  e(.  such 
that  the  net  controller  is  defined  by: 

C(q.q.t)  -  ][>,•  +  e, )<?;(</. qJ) 

i 

In  summary,  the  idea  is  that  the  force  field  described  by  the  biological  motor  controller  of  Eq. 
(12)  may  be  implemented  by  the  nervous  system  through  a  weighted  sum  of  motor  compu¬ 
tational  elements,  with  each  element  representing  the  time-varying  force  field  produced  by  a 
spinal  circuitry. 

A  relevant  feature  of  these  motor  computational  elements  is  their  convergence  toward  an 
equilibrium  location  within  the  limb's  reachable  space,  i.e..  a  location  where  forces  go  to  zero 
(Giszter  et  al.  1993).  Because  the  output  force  is  non-zero  for  the  rest  of  the  workspace. 
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the  "motor  receptive  field”  of  each  element  is.  by  definition,  rather  wide.  This  is  unlike  the 
properties  of  elements  in  early  vision  which  are  involved  in  learning,  a  hyperacuity  task  ( Poggio 
et  al.  1992).  It  follows  that  if  a  learning  task  required  approximation  of  a  force  field,  the  broad 
motor  receptive  fields  of  the  computational  elements  should  lead  to  generalization  well  beyond 
the  neighborhood  of  the  training  data.  To  test  this  idea,  we  limited  the  input  space  for  which 
training  data  were  provided  and  tested  the  nervous  system's  ability  to  generalize  to  a  region 
outside  of  the  boundary  of  the  training  data.  Our  scenario  was  one  in  which  information  about 
the  environment  was  available  for  only  a  small  part  of  the  workspace:  this  was  achieved  by 
limiting  the  region  where  the  subject  practiced  a  movement.  Once  adaptation  had  occurred, 
the  subject  performed  movements  in  a  novel  region  where  no  training  had  been  provided.  We 
found  that  exposure  to  an  environment  resulted  in  after  effects  beyond  the  novel  region,  i.e.. 
there  was  a  transferred  after  effect  (Fig.  13B).  This  indicated  that  the  internal  model  was  used 
by  the  nervous  system  to  predict  behavior  of  the  environment  well  beyond  the  boundary  of  the 
training  region. 

Front  the  after-effects  at  the  novel  region  we  can  state  that  the  computational  elements  with 
which  the  nervous  system  formed  a  model  of  the  environmental  forces  had  wide  motor  receptive 
fields.  These  elements  produced  a  significant  response  for  a  region  of  the  workspace  well  beyond 
tin  neighborhood  where  training  data  were  provided.  This  property  of  the  adaptive  controller  is 
inconsistent  with  the  approach  where  motor  learning  takes  place  via  construction  of  a  look-  up 
table  in  which  local  association  is  made  between  visited  states  (address  of  the  memory  cells 
in  the  table)  and  experienced  forces  (contents  of  the  cells).  On  the  contrary,  adaptation  is 
via  computational  elements  which  give  the  property  of  generalization  to  the  composed  internal 
model. 

The  after-effects  at  the  left  workspace  suggest  that  the  internal  model  generalized  the 
environmental  forces  to  a  specific  pattern.  Interestingly,  from  the  trajectory  of  after-effects 
(Fig.  13B).  it  was  apparent  that  the  expected  force  field  at  the  left  workspace  was  very  different 
than  the  one  that  the  subject  had  been  trained  on.  We  hypothesized  that  this  difference  could 
be  accounted  for  if  the  field  was  generalized  not  in  terms  of  forces  on  the  hand,  but  in  terms 
of  torques  on  the  joints.  The  idea  was  that  perhaps  the  relative  position  of  the  computational 
elements  in  the  motor  control  hierarchy  dictated  the  coordinate  system  in  which  information 
about  the  environment  was  generalized:  if  these  elements  resided  near  the  plan  stage  of  the  task, 
w'here  a  desired  hand  trajectory  is  specified,  then  they  might  encode  the  environmental  dynamics 
as  a  mapping  between  the  state  of  the  arm  and  imposed  forces  in  an  extrinsic  frame  of  reference. 
Assuming  that  these  elements  broadly  encoded  the  input  space,  then  local  adaptation  might 
produce  an  internal  model  which  generalized  to  similar  endpoint  forces  for  similar  endpoint 
trajectories.  Alternatively,  the  computational  elements  might  reside  at  a  lower  stage,  perhaps 
near  the  effectors,  wdtere  information  is  received  in  a  coordinate  system  defined  by  the  afferents 
and  the  muscles.  Here  the  internal  model  would  be  a  mapping  between  observed  states  of  the 
arm  and  the  imposed  forces  in  an  intrinsic  frame  of  reference.  As  opposed  to  the  high  level 
model,  local  adaptation  here  might  produce  a  map  which  generalizes  to  similar  joint  torques 
for  similar  joint  trajectories. 

We  tested  the  merits  of  these  alternate  scenarios  by  a  direct  experiment  .  After  practicing 
in  a  field  at  the  right  workspace,  the  subjects  were  asked  to  make  reaching  movements  at  the 
left  workspace.  The  field  presented  at  the  novel  region  (left  workspace)  was  one  of  two  kinds. 
In  some  trials,  this  field  was  a  translation  of  the  training  field  in  endpoint  coordinates,  while  in 
the  other  trials  the  field  presented  was  a  translation  of  the  training  field  in  joint  coordinates. 
Consistently  w'e  found  that  the  performance  of  the  subjects  was  near  optimum  when  the  field 
wras  translated  in  joint  coordinates  (Figs.  14A  and  15).  This  finding  is  in  sharp  contrast  with 
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t lie  hypothesis  that  subjects  adapted  to  the  imposed  field  by  building  a  model  in  endpoint 
coordinates.  On  the  contrary,  our  finding  suggests  that  the  subjects  represented  the  imposed 
force  field  as  a  map  between  motion  and  forces  in  the  intrinsic  coordinate  system  of  the  afferent s 
and  actuators. 

In  conclusion,  this  choice  of  the  coordinate  system  for  the  internal  model  suggests  that 
the  planning  and  control  of  a  reaching  movement  are  undertaken  by  fundamentally  different 
computational  elements  in  the  nervous  system.  While  the  planned  trajectory  for  the  arm  is 
in  an  extrinsic  frame  of  reference,  the  model  for  the  dynamics  of  the  task  is  in  an  intrinsic 
frame.  What  results  is  a  scenario  where  learning  a  motor  task,  say  hitting  a  golf  ball,  entails 
both  formation  of  an  appropriate  kinematic  plan.  i.e..  golf  club  trajectory,  and  composition  of  a 
model  of  the  task's  dynamics  so  that  the  plan  may  be  executed,  i.e..  forming  an  internal  model 
of  the  club's  dynamics.  Here  we  have  reported  on  some  ol  the  properties  of  the  computational 
elements  with  which  the  nervous  system  forms  the  internal  model  for  task's  dynamics.  It 
remains  to  be  seen  whether  computational  elements  which  are  involved  in  learning  kinematics 
of  a  task  produce  a  model  which  fundamentally  has  a  different  geometric  property  than  that 
which  results  while  learning  dynamics.  Perhaps  eventually  the  combination  of  elements  involved 
in  learning  kinematics  an  dynamics  can  form  a  kind  of  alphabet  for  the  language  of  movement. 


Acknowledgements:  This  work  has  been  greatly  enriched  because  of  our  interactions  with  Drs.  Emilio 
Bizzi,  Simon  (liszter,  Richard  Held,  Neville  Hogan.  Mike  Jordan.  Eric  Loeb.  and  Tomaso  Poggio.  We 
are  particularly  grateful  for  the  time  and  attention  given  to  this  project  by  Prof.  Bizzi. 

Send  Correspondence  to:  Dr.  Reza  Shadmehr.  Room  E25--201.  Dept,  of  Brain  and  Cognitive  Sciences. 
Massachusetts  Institute  of  Technology.  Cambridge,  M A  02139  USA.  Email:  rezanai.mit.edu 


29 


Appendix  I:  Correlation  ol  two  sampled  vector  fields 


Here  we  describe  a  mathematical  technique  that  quantifies  the  similarity  between  two  sampled 
tields.  We  used  this  technique  lor  comparing  force  fields  as  well  as  hand  trajectories.  This 
technique  was  based  on  the  notion  of  inner  product  of  two  sampler!  vector  fields  (Candolfo  and 
Mussa-Ivaldi  1993). 

Empirically,  a  time  series  of  vectors,  as  well  as  a  vector  field,  may  be  regarded  as  a  finite 
ordered  set  of  vectors,  sampled  at  subsequent  instance  of  time,  or  in  a  given  arrangement  of 
spatial  locations.  In  mathematical  terms,  a  finite  ordered  set  of  vectors.  I  .  is  a  mapping  that 

assigns  to  each  element,  i.  of  the  index  set.  jl . n)  G  A  .  a  vector  //,.  1  hen  the  expected 

value  of  C.  denoted  by  5(C).  is  a  mapping  from  the  same  index  set  to  the  set  of  vectors  {c,}. 
where 


According  to  this  definition,  the  expected  value  of  /  is  a  constant  set  ( r,  =  t'j.Vi.j).  It 
follows  that:  z(s(l))  =  s{l  ).  Now  consider  the  task  of  comparing  two  sets  (  and  V.  where 
Y  =  (y\.ij2 . yn).  Let  us  define  the  inner  product  of  V  and  V  as  the  scalar: 


n 

<  r.  y  >  =  ",  ■  y, 

i=i 

where  the  symbol  •  on  the  right  side  indicates  the  dot  product  operation  between  two  vectors. 
We  define  the  expected  value  of  this  inner  product  as: 

s(<  i  .y  >)  -  -  <  r. v  > 

n 

Then,  we  may  use  the  above  expressions  for  defining  the  co- variance  of  two  vectorial  sets: 


Co x(U.Y)  =  s«  T  -  s([  ).y  -  s(Y)  >) 

=  s(<  r.y  >)-  <  z(n.s(Y)  > 


Furthermore,  the  correlation  coefficient  between  two  sets.  p{l'.Y).  is  given  by  the  ratio  of  the 
co- variance  of  the  time  series  and  the  product  of  their  standard  deviations: 


n(i\Y) 


Co \{L  '.Y ) 

a(V)o(Y) 


where  standard  deviation  of  an  ordered  set  of  vectors  is  the  scalar: 

air)  =  c(iir  -5(nii)1/2 

and  ||C  ||  is  defined  as:  ||C||  =  (<  l\V  >)1/2.  It  follows  that  —  1  <  p(l  .  I  )  <  +1. 
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